[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:31:31 dist-test-slave-266d krb5kdc[32105](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:31:31 dist-test-slave-266d krb5kdc[32105](info): set up 2 sockets
Aug 08 22:31:31 dist-test-slave-266d krb5kdc[32105](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260808 22:31:32.440892 32094 mini_kdc.cc:121] Time spent starting KDC: real 1.371s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Aug 08 22:31:32 dist-test-slave-266d krb5kdc[32105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228292, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:31:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:31:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.87.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
I20260808 22:31:32.563156 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:46865
--webserver_interface=127.31.87.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:33463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:46865
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:31:32.681715 32123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:32.681967 32123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:32.682018 32123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:32.685909 32123 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:31:32.685963 32123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:32.685989 32123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:32.686034 32123 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:31:32.686069 32123 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:31:32.691406 32123 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:33463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:46865
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:46865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.87.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32123
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:32.692727 32123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:32.694032 32123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:32.704890 32130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:32.706508 32135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:32.706672 32123 server_base.cc:1034] running on GCE node
W20260808 22:31:32.706786 32133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:32.707370 32123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:32.708981 32123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:32.710160 32123 hybrid_clock.cc:648] HybridClock initialized: now 1786228292710133 us; error 35 us; skew 500 ppm
Aug 08 22:31:32 dist-test-slave-266d krb5kdc[32105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228292, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:31:32.712952 32123 init.cc:377] Logged in from keytab as kudu/127.31.87.190@KRBTEST.COM (short username kudu)
I20260808 22:31:32.714195 32123 webserver.cc:460] Webserver started at http://127.31.87.190:40005/ using document root <none> and password file <none>
I20260808 22:31:32.714752 32123 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:32.714819 32123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:32.715113 32123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:32.717291 32123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/master-0/data/instance:
uuid: "3779e73b86c845cfabbeed5ec2381478"
format_stamp: "Formatted at 2026-08-08 22:31:32 on dist-test-slave-266d"
server_key: "b69a6217e2fe49110d1be325e835ea26"
server_key_iv: "9100c2a8857de4542da18e631d5d8e82"
server_key_version: "encryptionkey@0"
I20260808 22:31:32.717837 32123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/master-0/wal/instance:
uuid: "3779e73b86c845cfabbeed5ec2381478"
format_stamp: "Formatted at 2026-08-08 22:31:32 on dist-test-slave-266d"
server_key: "b69a6217e2fe49110d1be325e835ea26"
server_key_iv: "9100c2a8857de4542da18e631d5d8e82"
server_key_version: "encryptionkey@0"
I20260808 22:31:32.722007 32123 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 22:31:32.724418 32146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:32.725688 32123 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:31:32.725893 32123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "3779e73b86c845cfabbeed5ec2381478"
format_stamp: "Formatted at 2026-08-08 22:31:32 on dist-test-slave-266d"
server_key: "b69a6217e2fe49110d1be325e835ea26"
server_key_iv: "9100c2a8857de4542da18e631d5d8e82"
server_key_version: "encryptionkey@0"
I20260808 22:31:32.725998 32123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:32.755667 32123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:32.758747 32123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:32.758947 32123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:32.767917 32123 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.190:46865
I20260808 22:31:32.767966 32230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.190:46865 every 8 connection(s)
I20260808 22:31:32.769209 32123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:31:32.773056 32231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:32.779680 32231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3779e73b86c845cfabbeed5ec2381478: Bootstrap starting.
I20260808 22:31:32.779819 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 32123
I20260808 22:31:32.779960 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/master-0/wal/instance
I20260808 22:31:32.781275 32094 external_mini_cluster.cc:1428] Setting key 9cb0483dc8d4633b2731c90fc21fc00c
I20260808 22:31:32.782409 32231 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3779e73b86c845cfabbeed5ec2381478: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:32.783396 32231 log.cc:824] T 00000000000000000000000000000000 P 3779e73b86c845cfabbeed5ec2381478: Log is configured to *not* fsync() on all Append() calls
I20260808 22:31:32.786152 32231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3779e73b86c845cfabbeed5ec2381478: No bootstrap required, opened a new log
I20260808 22:31:32.790912 32231 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3779e73b86c845cfabbeed5ec2381478 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3779e73b86c845cfabbeed5ec2381478" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 46865 } }
I20260808 22:31:32.791188 32231 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3779e73b86c845cfabbeed5ec2381478 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:32.791288 32231 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3779e73b86c845cfabbeed5ec2381478 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3779e73b86c845cfabbeed5ec2381478, State: Initialized, Role: FOLLOWER
I20260808 22:31:32.791975 32231 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3779e73b86c845cfabbeed5ec2381478 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3779e73b86c845cfabbeed5ec2381478" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 46865 } }
I20260808 22:31:32.792181 32231 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3779e73b86c845cfabbeed5ec2381478 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:31:32.792311 32231 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3779e73b86c845cfabbeed5ec2381478 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:31:32.792445 32231 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3779e73b86c845cfabbeed5ec2381478 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:32.793372 32231 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3779e73b86c845cfabbeed5ec2381478 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3779e73b86c845cfabbeed5ec2381478" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 46865 } }
I20260808 22:31:32.793859 32231 leader_election.cc:302] T 00000000000000000000000000000000 P 3779e73b86c845cfabbeed5ec2381478 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3779e73b86c845cfabbeed5ec2381478; no voters:
I20260808 22:31:32.794205 32231 leader_election.cc:288] T 00000000000000000000000000000000 P 3779e73b86c845cfabbeed5ec2381478 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:31:32.794335 32237 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3779e73b86c845cfabbeed5ec2381478 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:31:32.794515 32237 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3779e73b86c845cfabbeed5ec2381478 [term 1 LEADER]: Becoming Leader. State: Replica: 3779e73b86c845cfabbeed5ec2381478, State: Running, Role: LEADER
I20260808 22:31:32.794911 32237 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3779e73b86c845cfabbeed5ec2381478 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3779e73b86c845cfabbeed5ec2381478" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 46865 } }
I20260808 22:31:32.795799 32231 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3779e73b86c845cfabbeed5ec2381478 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:31:32.797441 32239 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3779e73b86c845cfabbeed5ec2381478 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3779e73b86c845cfabbeed5ec2381478. Latest consensus state: current_term: 1 leader_uuid: "3779e73b86c845cfabbeed5ec2381478" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3779e73b86c845cfabbeed5ec2381478" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 46865 } } }
I20260808 22:31:32.797575 32239 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3779e73b86c845cfabbeed5ec2381478 [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:32.798039 32238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3779e73b86c845cfabbeed5ec2381478 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3779e73b86c845cfabbeed5ec2381478" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3779e73b86c845cfabbeed5ec2381478" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 46865 } } }
I20260808 22:31:32.798138 32238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3779e73b86c845cfabbeed5ec2381478 [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:32.800124 32251 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 08 22:31:32 dist-test-slave-266d krb5kdc[32105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228292, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:31:32.803843 32251 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:31:32.809263 32234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:32.785679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53932 (local address 127.31.87.190:46865)
0808 22:31:32.786096 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:32.786113 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:32.787481 (+ 1368us) server_negotiation.cc:407] Connection header received
0808 22:31:32.790382 (+ 2901us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:32.790396 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:32.790784 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:32.791091 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:32.793483 (+ 2392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:32.794619 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:32.795944 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:32.796418 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:32.796576 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:32.802022 (+ 5446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:32.802066 (+ 44us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:32.802082 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:32.802114 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:32.804250 (+ 2136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:32.804914 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:32.804920 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:32.804926 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:32.805077 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:32.805654 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:32.805658 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:32.805660 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:32.806309 (+ 649us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:32.806511 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:32.807148 (+ 637us) server_negotiation.cc:299] Negotiation successful
0808 22:31:32.807436 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":105,"threads_started":1}
I20260808 22:31:32.811189 32251 catalog_manager.cc:1347] Generated new cluster ID: 2da0c9461998402891969de058023980
I20260808 22:31:32.811251 32251 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:31:32.827723 32251 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:31:32.828668 32251 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:31:32.837363 32251 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3779e73b86c845cfabbeed5ec2381478: Generated new TSK 0
I20260808 22:31:32.838351 32251 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.87.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab.
Entry for principal kudu/127.31.87.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab.
I20260808 22:31:32.867256 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.129:0
--local_ip_for_outbound_sockets=127.31.87.129
--webserver_interface=127.31.87.129
--webserver_port=0
--tserver_master_addrs=127.31.87.190:46865
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab
--principal=kudu/127.31.87.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:33463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:31:32.973826 32267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:32.974037 32267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:32.974076 32267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:32.977890 32267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:32.977934 32267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:32.977998 32267 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.129
I20260808 22:31:32.982932 32267 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:33463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab
--principal=kudu/127.31.87.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.87.129
--webserver_port=0
--tserver_master_addrs=127.31.87.190:46865
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32267
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.129
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:32.983826 32267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:32.984694 32267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:32.994095 32273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:32.994127 32274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:32.994197 32267 server_base.cc:1034] running on GCE node
W20260808 22:31:32.994143 32279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:32.994637 32267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:32.995221 32267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:32.996356 32267 hybrid_clock.cc:648] HybridClock initialized: now 1786228292996341 us; error 25 us; skew 500 ppm
Aug 08 22:31:33 dist-test-slave-266d krb5kdc[32105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228293, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:31:33.002139 32267 init.cc:377] Logged in from keytab as kudu/127.31.87.129@KRBTEST.COM (short username kudu)
I20260808 22:31:33.002911 32267 webserver.cc:460] Webserver started at http://127.31.87.129:44711/ using document root <none> and password file <none>
I20260808 22:31:33.003417 32267 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:33.003471 32267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:33.003625 32267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:33.005595 32267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/ts-0/data/instance:
uuid: "ae104a14260c462c8398976d0bc71790"
format_stamp: "Formatted at 2026-08-08 22:31:33 on dist-test-slave-266d"
server_key: "5a612c18b2cc758ce6a7c07323d07cec"
server_key_iv: "2c4c4cfbe136b19e01aa9ba73d66edca"
server_key_version: "encryptionkey@0"
I20260808 22:31:33.006088 32267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/ts-0/wal/instance:
uuid: "ae104a14260c462c8398976d0bc71790"
format_stamp: "Formatted at 2026-08-08 22:31:33 on dist-test-slave-266d"
server_key: "5a612c18b2cc758ce6a7c07323d07cec"
server_key_iv: "2c4c4cfbe136b19e01aa9ba73d66edca"
server_key_version: "encryptionkey@0"
I20260808 22:31:33.009722 32267 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 22:31:33.012351 32288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:33.013598 32267 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:31:33.013701 32267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/ts-0/wal
uuid: "ae104a14260c462c8398976d0bc71790"
format_stamp: "Formatted at 2026-08-08 22:31:33 on dist-test-slave-266d"
server_key: "5a612c18b2cc758ce6a7c07323d07cec"
server_key_iv: "2c4c4cfbe136b19e01aa9ba73d66edca"
server_key_version: "encryptionkey@0"
I20260808 22:31:33.013797 32267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:33.027560 32267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:33.030507 32267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:33.030684 32267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:33.031355 32267 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:31:33.032503 32267 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:31:33.032557 32267 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:33.032617 32267 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:31:33.032644 32267 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:33.042299 32267 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.129:36851
I20260808 22:31:33.042362 32474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.129:36851 every 8 connection(s)
I20260808 22:31:33.043192 32267 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
I20260808 22:31:33.052896 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 32267
I20260808 22:31:33.053009 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/ts-0/wal/instance
Aug 08 22:31:33 dist-test-slave-266d krb5kdc[32105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228293, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.129@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:31:33.053323 32094 external_mini_cluster.cc:1428] Setting key 704b063298e65fa6cc8dea5909fa56c6
I20260808 22:31:33.057331 32234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:33.045440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:55805 (local address 127.31.87.190:46865)
0808 22:31:33.045605 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:33.045610 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:33.046476 (+ 866us) server_negotiation.cc:407] Connection header received
0808 22:31:33.047931 (+ 1455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:33.047934 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:33.047981 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:33.048055 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:33.049246 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:33.049815 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:33.050942 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:33.051190 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:33.051278 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:33.053782 (+ 2504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:33.053785 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:33.053788 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:33.053810 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:33.055289 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:33.055796 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:33.055801 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:33.055803 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:33.055897 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:33.056225 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:33.056228 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:33.056229 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:33.056453 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:33.056557 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:33.057010 (+ 453us) server_negotiation.cc:299] Negotiation successful
0808 22:31:33.057153 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 22:31:33.058275 32480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:33.045837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:46865 (local address 127.31.87.129:55805)
0808 22:31:33.046308 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:33.046351 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:33.047708 (+ 1357us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:33.048214 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:33.048220 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:33.048594 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:33.049058 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:33.049072 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:33.049943 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:33.049946 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:33.050828 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:33.050838 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:33.051313 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:33.051316 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:33.051448 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:33.051979 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:31:33.051997 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:33.053611 (+ 1614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:33.055402 (+ 1791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:33.055407 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:33.055417 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:33.055685 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:33.055986 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:33.055988 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:33.055989 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:33.056141 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:33.056553 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:33.056560 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:33.056784 (+ 224us) client_negotiation.cc:769] Sending connection context
0808 22:31:33.056944 (+ 160us) client_negotiation.cc:240] Negotiation successful
0808 22:31:33.057276 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":122,"threads_started":1}
I20260808 22:31:33.059501 32476 heartbeater.cc:347] Connected to a master server at 127.31.87.190:46865
I20260808 22:31:33.059780 32476 heartbeater.cc:464] Registering TS with master...
I20260808 22:31:33.060380 32476 heartbeater.cc:511] Master 127.31.87.190:46865 requested a full tablet report, sending...
I20260808 22:31:33.062403 32175 ts_manager.cc:194] Registered new tserver with Master: ae104a14260c462c8398976d0bc71790 (127.31.87.129:36851)
I20260808 22:31:33.063812 32175 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.129@KRBTEST.COM'} at 127.31.87.129:55805
WARNING: no policy specified for kudu/127.31.87.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab.
Entry for principal kudu/127.31.87.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab.
I20260808 22:31:33.079449 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.130:0
--local_ip_for_outbound_sockets=127.31.87.130
--webserver_interface=127.31.87.130
--webserver_port=0
--tserver_master_addrs=127.31.87.190:46865
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab
--principal=kudu/127.31.87.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:33463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:31:33.186154 32484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:33.186378 32484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:33.186435 32484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:33.190279 32484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:33.190344 32484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:33.190423 32484 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.130
I20260808 22:31:33.195273 32484 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:33463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab
--principal=kudu/127.31.87.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.87.130
--webserver_port=0
--tserver_master_addrs=127.31.87.190:46865
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.130
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:33.196162 32484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:33.197067 32484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:33.206331 32494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:33.206292 32497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:33.206341 32493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:33.206470 32484 server_base.cc:1034] running on GCE node
I20260808 22:31:33.206884 32484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:33.207420 32484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:33.208570 32484 hybrid_clock.cc:648] HybridClock initialized: now 1786228293208550 us; error 30 us; skew 500 ppm
Aug 08 22:31:33 dist-test-slave-266d krb5kdc[32105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228293, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:31:33.211050 32484 init.cc:377] Logged in from keytab as kudu/127.31.87.130@KRBTEST.COM (short username kudu)
I20260808 22:31:33.211833 32484 webserver.cc:460] Webserver started at http://127.31.87.130:44329/ using document root <none> and password file <none>
I20260808 22:31:33.212313 32484 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:33.212371 32484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:33.212554 32484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:33.214643 32484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/ts-1/data/instance:
uuid: "e684bd228da14ee3bd0ad94a5181a1a9"
format_stamp: "Formatted at 2026-08-08 22:31:33 on dist-test-slave-266d"
server_key: "09623c093cd9354f167213a0fc6971c0"
server_key_iv: "f83c20ebb036ad310385c1e353c58341"
server_key_version: "encryptionkey@0"
I20260808 22:31:33.215160 32484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/ts-1/wal/instance:
uuid: "e684bd228da14ee3bd0ad94a5181a1a9"
format_stamp: "Formatted at 2026-08-08 22:31:33 on dist-test-slave-266d"
server_key: "09623c093cd9354f167213a0fc6971c0"
server_key_iv: "f83c20ebb036ad310385c1e353c58341"
server_key_version: "encryptionkey@0"
I20260808 22:31:33.218837 32484 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 22:31:33.221041 32505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:33.222170 32484 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:31:33.222286 32484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/ts-1/wal
uuid: "e684bd228da14ee3bd0ad94a5181a1a9"
format_stamp: "Formatted at 2026-08-08 22:31:33 on dist-test-slave-266d"
server_key: "09623c093cd9354f167213a0fc6971c0"
server_key_iv: "f83c20ebb036ad310385c1e353c58341"
server_key_version: "encryptionkey@0"
I20260808 22:31:33.222388 32484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:33.256242 32484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:33.258939 32484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:33.259114 32484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:33.259682 32484 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:31:33.260494 32484 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:31:33.260542 32484 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:33.260594 32484 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:31:33.260620 32484 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:33.270454 32484 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.130:37501
I20260808 22:31:33.270478 32679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.130:37501 every 8 connection(s)
I20260808 22:31:33.271281 32484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
I20260808 22:31:33.274824 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 32484
I20260808 22:31:33.274931 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/ts-1/wal/instance
I20260808 22:31:33.275166 32094 external_mini_cluster.cc:1428] Setting key 2348162316f31f653c58398ad6435bea
Aug 08 22:31:33 dist-test-slave-266d krb5kdc[32105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228293, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.130@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.87.131@KRBTEST.COM; defaulting to no policy
I20260808 22:31:33.286116 32234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:33.273274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:53673 (local address 127.31.87.190:46865)
0808 22:31:33.273405 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:33.273409 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:33.274212 (+ 803us) server_negotiation.cc:407] Connection header received
0808 22:31:33.275541 (+ 1329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:33.275545 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:33.275598 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:33.275679 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:33.277000 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:33.277719 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:33.278850 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:33.279069 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:33.279154 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:33.282744 (+ 3590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:33.282748 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:33.282751 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:33.282779 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:33.284250 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:33.284752 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:33.284755 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:33.284757 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:33.284849 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:33.285192 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:33.285195 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:33.285197 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:33.285374 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:33.285450 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:33.285899 (+ 449us) server_negotiation.cc:299] Negotiation successful
0808 22:31:33.286000 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 22:31:33.286931 32682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:33.273656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:46865 (local address 127.31.87.130:53673)
0808 22:31:33.274070 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:33.274108 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:33.275338 (+ 1230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:33.275850 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:33.275858 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:33.276301 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:33.276854 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:33.276868 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:33.277840 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:33.277843 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:33.278729 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:33.278739 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:33.279198 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:33.279203 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:33.279351 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:33.281206 (+ 1855us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:31:33.281222 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:33.282580 (+ 1358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:33.284371 (+ 1791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:33.284378 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:33.284392 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:33.284658 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:33.284960 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:33.284963 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:33.284964 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:33.285114 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:33.285450 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:33.285455 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:33.285670 (+ 215us) client_negotiation.cc:769] Sending connection context
0808 22:31:33.285823 (+ 153us) client_negotiation.cc:240] Negotiation successful
0808 22:31:33.286108 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":120,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.131@KRBTEST.COM" created.
I20260808 22:31:33.287997 32680 heartbeater.cc:347] Connected to a master server at 127.31.87.190:46865
I20260808 22:31:33.288208 32680 heartbeater.cc:464] Registering TS with master...
I20260808 22:31:33.288640 32680 heartbeater.cc:511] Master 127.31.87.190:46865 requested a full tablet report, sending...
I20260808 22:31:33.289592 32175 ts_manager.cc:194] Registered new tserver with Master: e684bd228da14ee3bd0ad94a5181a1a9 (127.31.87.130:37501)
I20260808 22:31:33.290128 32175 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.130@KRBTEST.COM'} at 127.31.87.130:53673
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab.
Entry for principal kudu/127.31.87.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab.
I20260808 22:31:33.303176 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.131:0
--local_ip_for_outbound_sockets=127.31.87.131
--webserver_interface=127.31.87.131
--webserver_port=0
--tserver_master_addrs=127.31.87.190:46865
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab
--principal=kudu/127.31.87.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:33463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:31:33.411988 32686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:33.412252 32686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:33.412317 32686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:33.416263 32686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:33.416332 32686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:33.416409 32686 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.131
I20260808 22:31:33.421584 32686 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:33463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab
--principal=kudu/127.31.87.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.87.131
--webserver_port=0
--tserver_master_addrs=127.31.87.190:46865
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32686
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.131
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:33.422514 32686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:33.423442 32686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:33.432651 32694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:33.432798 32686 server_base.cc:1034] running on GCE node
W20260808 22:31:33.432659 32697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:33.432729 32693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:33.433305 32686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:33.433853 32686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:33.435006 32686 hybrid_clock.cc:648] HybridClock initialized: now 1786228293434989 us; error 30 us; skew 500 ppm
Aug 08 22:31:33 dist-test-slave-266d krb5kdc[32105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228293, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:31:33.437532 32686 init.cc:377] Logged in from keytab as kudu/127.31.87.131@KRBTEST.COM (short username kudu)
I20260808 22:31:33.438320 32686 webserver.cc:460] Webserver started at http://127.31.87.131:41677/ using document root <none> and password file <none>
I20260808 22:31:33.438833 32686 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:33.438897 32686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:33.439083 32686 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:33.440999 32686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/ts-2/data/instance:
uuid: "717a071cacf245d2965568f7ea46233f"
format_stamp: "Formatted at 2026-08-08 22:31:33 on dist-test-slave-266d"
server_key: "7bb65cedb481afebd5268ca6da4f23dd"
server_key_iv: "e09410f5e554f0c7b2b8779b3b798d0d"
server_key_version: "encryptionkey@0"
I20260808 22:31:33.441560 32686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/ts-2/wal/instance:
uuid: "717a071cacf245d2965568f7ea46233f"
format_stamp: "Formatted at 2026-08-08 22:31:33 on dist-test-slave-266d"
server_key: "7bb65cedb481afebd5268ca6da4f23dd"
server_key_iv: "e09410f5e554f0c7b2b8779b3b798d0d"
server_key_version: "encryptionkey@0"
I20260808 22:31:33.445186 32686 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:31:33.447394 32711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:33.448520 32686 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:31:33.448632 32686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/ts-2/wal
uuid: "717a071cacf245d2965568f7ea46233f"
format_stamp: "Formatted at 2026-08-08 22:31:33 on dist-test-slave-266d"
server_key: "7bb65cedb481afebd5268ca6da4f23dd"
server_key_iv: "e09410f5e554f0c7b2b8779b3b798d0d"
server_key_version: "encryptionkey@0"
I20260808 22:31:33.448720 32686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:33.485348 32686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:33.487983 32686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:33.488157 32686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:33.488734 32686 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:31:33.489588 32686 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:31:33.489662 32686 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:33.489727 32686 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:31:33.489753 32686 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:33.501317 32686 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.131:36291
I20260808 22:31:33.501360 417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.131:36291 every 8 connection(s)
I20260808 22:31:33.502174 32686 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
I20260808 22:31:33.509586 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 32686
I20260808 22:31:33.509698 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786228291013016-32094-0/minicluster-data/ts-2/wal/instance
I20260808 22:31:33.509961 32094 external_mini_cluster.cc:1428] Setting key 519c76c79eab85c1ff0ca68cf06509f7
Aug 08 22:31:33 dist-test-slave-266d krb5kdc[32105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228293, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.131@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:31:33.514938 32234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:33.503870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:33625 (local address 127.31.87.190:46865)
0808 22:31:33.504027 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:33.504031 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:33.504825 (+ 794us) server_negotiation.cc:407] Connection header received
0808 22:31:33.506151 (+ 1326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:33.506154 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:33.506203 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:33.506280 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:33.507363 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:33.507901 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:33.508935 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:33.509170 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:33.509241 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:33.511616 (+ 2375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:33.511620 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:33.511622 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:33.511644 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:33.513041 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:33.513532 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:33.513536 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:33.513538 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:33.513621 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:33.513926 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:33.513929 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:33.513931 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:33.514105 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:33.514190 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:33.514664 (+ 474us) server_negotiation.cc:299] Negotiation successful
0808 22:31:33.514785 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 22:31:33.515650 421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:33.504262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:46865 (local address 127.31.87.131:33625)
0808 22:31:33.504680 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:33.504722 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:33.505951 (+ 1229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:33.506418 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:33.506425 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:33.506783 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:33.507228 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:33.507240 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:33.507990 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:33.507992 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:33.508832 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:33.508842 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:33.509271 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:33.509274 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:33.509409 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:33.509948 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:31:33.509964 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:33.511450 (+ 1486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:33.513168 (+ 1718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:33.513173 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:33.513188 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:33.513438 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:33.513705 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:33.513708 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:33.513710 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:33.513841 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:33.514200 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:33.514206 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:33.514430 (+ 224us) client_negotiation.cc:769] Sending connection context
0808 22:31:33.514588 (+ 158us) client_negotiation.cc:240] Negotiation successful
0808 22:31:33.514887 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":106,"threads_started":1}
I20260808 22:31:33.516597 418 heartbeater.cc:347] Connected to a master server at 127.31.87.190:46865
I20260808 22:31:33.516824 418 heartbeater.cc:464] Registering TS with master...
I20260808 22:31:33.517320 418 heartbeater.cc:511] Master 127.31.87.190:46865 requested a full tablet report, sending...
I20260808 22:31:33.518272 32175 ts_manager.cc:194] Registered new tserver with Master: 717a071cacf245d2965568f7ea46233f (127.31.87.131:36291)
I20260808 22:31:33.518817 32175 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.131@KRBTEST.COM'} at 127.31.87.131:33625
I20260808 22:31:33.524158 32094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 22:31:33 dist-test-slave-266d krb5kdc[32105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228293, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 22:31:33 dist-test-slave-266d krb5kdc[32105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228293, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.87.129@KRBTEST.COM
I20260808 22:31:33.551745 430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:33.543404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46782 (local address 127.31.87.129:36851)
0808 22:31:33.543703 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:33.543718 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:33.543778 (+ 60us) server_negotiation.cc:407] Connection header received
0808 22:31:33.543915 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:33.543920 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:33.544101 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:33.544231 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:33.544777 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:33.545569 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:33.546325 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:33.546669 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:33.546744 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:33.548827 (+ 2083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:33.548837 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:33.548848 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:33.548871 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:33.550330 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:33.550676 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:33.550680 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:33.550681 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:33.550751 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:33.550938 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:33.550940 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:33.550941 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:33.551148 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:33.551307 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:33.551435 (+ 128us) server_negotiation.cc:299] Negotiation successful
0808 22:31:33.551555 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":94,"threads_started":1}
W20260808 22:31:33.552696 32330 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46782
Aug 08 22:31:33 dist-test-slave-266d krb5kdc[32105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228293, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 22:31:33 dist-test-slave-266d krb5kdc[32105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228293, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.87.129@KRBTEST.COM
I20260808 22:31:33.576102 430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:33.569075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46786 (local address 127.31.87.129:36851)
0808 22:31:33.569219 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:33.569222 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:33.569399 (+ 177us) server_negotiation.cc:407] Connection header received
0808 22:31:33.569496 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:33.569499 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:33.569535 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:33.569599 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:33.570266 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:33.570752 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:33.571469 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:33.571680 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:33.571741 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:33.573780 (+ 2039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:33.573783 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:33.573785 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:33.573806 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:33.574889 (+ 1083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:33.575216 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:33.575218 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:33.575220 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:33.575279 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:33.575449 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:33.575450 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:33.575451 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:33.575654 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:33.575745 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:33.575905 (+ 160us) server_negotiation.cc:299] Negotiation successful
0808 22:31:33.575991 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260808 22:31:33.577055 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 32267
I20260808 22:31:33.589704 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 32484
I20260808 22:31:33.622061 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 32686
I20260808 22:31:33.638024 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 32123
2026-08-08T22:31:33Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2603 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:31:33 dist-test-slave-266d krb5kdc[439](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:31:33 dist-test-slave-266d krb5kdc[439](info): set up 2 sockets
Aug 08 22:31:33 dist-test-slave-266d krb5kdc[439](info): commencing operation
krb5kdc: starting...
W20260808 22:31:34.773934 32094 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Aug 08 22:31:34 dist-test-slave-266d krb5kdc[439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:31:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:31:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.87.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
I20260808 22:31:34.886312 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:40785
--webserver_interface=127.31.87.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:33253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:40785
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:31:34.992204 457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:34.992432 457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:34.992475 457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:34.996363 457 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:31:34.996411 457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:34.996431 457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:34.996464 457 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:31:34.996490 457 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:31:35.001508 457 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:33253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:40785
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:40785
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.87.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:35.002405 457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:35.003320 457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:35.011139 466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:35.011279 457 server_base.cc:1034] running on GCE node
W20260808 22:31:35.011353 469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:35.011408 465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:35.011900 457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:35.012883 457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:35.014038 457 hybrid_clock.cc:648] HybridClock initialized: now 1786228295014009 us; error 42 us; skew 500 ppm
Aug 08 22:31:35 dist-test-slave-266d krb5kdc[439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228295, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:31:35.016484 457 init.cc:377] Logged in from keytab as kudu/127.31.87.190@KRBTEST.COM (short username kudu)
I20260808 22:31:35.017320 457 webserver.cc:460] Webserver started at http://127.31.87.190:39237/ using document root <none> and password file <none>
I20260808 22:31:35.017827 457 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:35.017889 457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:35.018064 457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:35.019913 457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/master-0/data/instance:
uuid: "d1ac9ea0704c460f968dc24abbdf737d"
format_stamp: "Formatted at 2026-08-08 22:31:35 on dist-test-slave-266d"
server_key: "c971f581f87a18f9fc67704a75f8dcd2"
server_key_iv: "0d7efa2ba00ebc9c7eee4f7d2d6a7961"
server_key_version: "encryptionkey@0"
I20260808 22:31:35.020423 457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/master-0/wal/instance:
uuid: "d1ac9ea0704c460f968dc24abbdf737d"
format_stamp: "Formatted at 2026-08-08 22:31:35 on dist-test-slave-266d"
server_key: "c971f581f87a18f9fc67704a75f8dcd2"
server_key_iv: "0d7efa2ba00ebc9c7eee4f7d2d6a7961"
server_key_version: "encryptionkey@0"
I20260808 22:31:35.023908 457 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 22:31:35.026156 486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:35.027242 457 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:31:35.027351 457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "d1ac9ea0704c460f968dc24abbdf737d"
format_stamp: "Formatted at 2026-08-08 22:31:35 on dist-test-slave-266d"
server_key: "c971f581f87a18f9fc67704a75f8dcd2"
server_key_iv: "0d7efa2ba00ebc9c7eee4f7d2d6a7961"
server_key_version: "encryptionkey@0"
I20260808 22:31:35.027441 457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:35.055248 457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:35.057783 457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:35.057948 457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:35.064541 457 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.190:40785
I20260808 22:31:35.064553 570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.190:40785 every 8 connection(s)
I20260808 22:31:35.065439 457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:31:35.068015 572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:35.071650 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 457
I20260808 22:31:35.071765 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/master-0/wal/instance
I20260808 22:31:35.072026 32094 external_mini_cluster.cc:1428] Setting key e35bdfabd25032d3d64d5a605fd2f6f8
I20260808 22:31:35.073668 572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1ac9ea0704c460f968dc24abbdf737d: Bootstrap starting.
I20260808 22:31:35.075726 572 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d1ac9ea0704c460f968dc24abbdf737d: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:35.076413 572 log.cc:824] T 00000000000000000000000000000000 P d1ac9ea0704c460f968dc24abbdf737d: Log is configured to *not* fsync() on all Append() calls
I20260808 22:31:35.078176 572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1ac9ea0704c460f968dc24abbdf737d: No bootstrap required, opened a new log
Aug 08 22:31:35 dist-test-slave-266d krb5kdc[439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:31:35.082067 572 raft_consensus.cc:348] T 00000000000000000000000000000000 P d1ac9ea0704c460f968dc24abbdf737d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1ac9ea0704c460f968dc24abbdf737d" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 40785 } }
I20260808 22:31:35.082265 572 raft_consensus.cc:374] T 00000000000000000000000000000000 P d1ac9ea0704c460f968dc24abbdf737d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:35.082304 572 raft_consensus.cc:729] T 00000000000000000000000000000000 P d1ac9ea0704c460f968dc24abbdf737d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1ac9ea0704c460f968dc24abbdf737d, State: Initialized, Role: FOLLOWER
I20260808 22:31:35.082681 572 consensus_queue.cc:260] T 00000000000000000000000000000000 P d1ac9ea0704c460f968dc24abbdf737d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1ac9ea0704c460f968dc24abbdf737d" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 40785 } }
I20260808 22:31:35.082818 572 raft_consensus.cc:388] T 00000000000000000000000000000000 P d1ac9ea0704c460f968dc24abbdf737d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:31:35.082886 572 raft_consensus.cc:482] T 00000000000000000000000000000000 P d1ac9ea0704c460f968dc24abbdf737d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:31:35.082983 572 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d1ac9ea0704c460f968dc24abbdf737d [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:35.083848 572 raft_consensus.cc:504] T 00000000000000000000000000000000 P d1ac9ea0704c460f968dc24abbdf737d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1ac9ea0704c460f968dc24abbdf737d" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 40785 } }
I20260808 22:31:35.084172 572 leader_election.cc:302] T 00000000000000000000000000000000 P d1ac9ea0704c460f968dc24abbdf737d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d1ac9ea0704c460f968dc24abbdf737d; no voters:
I20260808 22:31:35.084486 572 leader_election.cc:288] T 00000000000000000000000000000000 P d1ac9ea0704c460f968dc24abbdf737d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:31:35.084622 578 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d1ac9ea0704c460f968dc24abbdf737d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:31:35.084820 578 raft_consensus.cc:686] T 00000000000000000000000000000000 P d1ac9ea0704c460f968dc24abbdf737d [term 1 LEADER]: Becoming Leader. State: Replica: d1ac9ea0704c460f968dc24abbdf737d, State: Running, Role: LEADER
I20260808 22:31:35.085165 578 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1ac9ea0704c460f968dc24abbdf737d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1ac9ea0704c460f968dc24abbdf737d" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 40785 } }
I20260808 22:31:35.085500 575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:35.073531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51440 (local address 127.31.87.190:40785)
0808 22:31:35.073947 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:35.073961 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:35.073997 (+ 36us) server_negotiation.cc:407] Connection header received
0808 22:31:35.074586 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:35.074619 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:35.074916 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:35.075202 (+ 286us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:35.075977 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:35.076983 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:35.077830 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:35.078285 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:35.078437 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:35.081028 (+ 2591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:35.081033 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:35.081045 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:35.081070 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:35.082699 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:35.083166 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:35.083171 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:35.083175 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:35.083282 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:35.083545 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:35.083547 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:35.083548 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:35.083886 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:35.084061 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:35.084307 (+ 246us) server_negotiation.cc:299] Negotiation successful
0808 22:31:35.084587 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":127,"threads_started":1}
I20260808 22:31:35.085795 572 sys_catalog.cc:564] T 00000000000000000000000000000000 P d1ac9ea0704c460f968dc24abbdf737d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:31:35.086750 581 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1ac9ea0704c460f968dc24abbdf737d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d1ac9ea0704c460f968dc24abbdf737d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1ac9ea0704c460f968dc24abbdf737d" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 40785 } } }
I20260808 22:31:35.086870 581 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1ac9ea0704c460f968dc24abbdf737d [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:35.087234 591 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:31:35.087186 582 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1ac9ea0704c460f968dc24abbdf737d [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1ac9ea0704c460f968dc24abbdf737d. Latest consensus state: current_term: 1 leader_uuid: "d1ac9ea0704c460f968dc24abbdf737d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1ac9ea0704c460f968dc24abbdf737d" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 40785 } } }
I20260808 22:31:35.087283 582 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1ac9ea0704c460f968dc24abbdf737d [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:35.090718 591 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:31:35.096405 591 catalog_manager.cc:1347] Generated new cluster ID: 592eb86727f742e68bff26c3790415ef
I20260808 22:31:35.096469 591 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:31:35.120214 591 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:31:35.121137 591 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:31:35.126525 591 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d1ac9ea0704c460f968dc24abbdf737d: Generated new TSK 0
I20260808 22:31:35.127231 591 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.87.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab.
Entry for principal kudu/127.31.87.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab.
I20260808 22:31:35.159865 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.129:0
--local_ip_for_outbound_sockets=127.31.87.129
--webserver_interface=127.31.87.129
--webserver_port=0
--tserver_master_addrs=127.31.87.190:40785
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab
--principal=kudu/127.31.87.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:33253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:31:35.272210 616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:35.272467 616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:35.272514 616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:35.276616 616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:35.276679 616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:35.276749 616 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.129
I20260808 22:31:35.281965 616 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:33253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab
--principal=kudu/127.31.87.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.87.129
--webserver_port=0
--tserver_master_addrs=127.31.87.190:40785
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.129
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:35.282967 616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:35.283933 616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:35.293077 629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:35.293193 625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:35.293241 626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:35.293277 616 server_base.cc:1034] running on GCE node
I20260808 22:31:35.293730 616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:35.294313 616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:35.295465 616 hybrid_clock.cc:648] HybridClock initialized: now 1786228295295443 us; error 33 us; skew 500 ppm
Aug 08 22:31:35 dist-test-slave-266d krb5kdc[439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228295, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:31:35.297988 616 init.cc:377] Logged in from keytab as kudu/127.31.87.129@KRBTEST.COM (short username kudu)
I20260808 22:31:35.298838 616 webserver.cc:460] Webserver started at http://127.31.87.129:46487/ using document root <none> and password file <none>
I20260808 22:31:35.299355 616 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:35.299417 616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:35.299597 616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:35.301540 616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/ts-0/data/instance:
uuid: "a029a9f6238c462a842b7dd0c133ba7c"
format_stamp: "Formatted at 2026-08-08 22:31:35 on dist-test-slave-266d"
server_key: "2d0ac78ecd2abca478a31f205e0710d0"
server_key_iv: "9704fe5565e75bdd6356c3c4de9d14a8"
server_key_version: "encryptionkey@0"
I20260808 22:31:35.302047 616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/ts-0/wal/instance:
uuid: "a029a9f6238c462a842b7dd0c133ba7c"
format_stamp: "Formatted at 2026-08-08 22:31:35 on dist-test-slave-266d"
server_key: "2d0ac78ecd2abca478a31f205e0710d0"
server_key_iv: "9704fe5565e75bdd6356c3c4de9d14a8"
server_key_version: "encryptionkey@0"
I20260808 22:31:35.305428 616 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 22:31:35.307621 637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:35.308701 616 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:31:35.308799 616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/ts-0/wal
uuid: "a029a9f6238c462a842b7dd0c133ba7c"
format_stamp: "Formatted at 2026-08-08 22:31:35 on dist-test-slave-266d"
server_key: "2d0ac78ecd2abca478a31f205e0710d0"
server_key_iv: "9704fe5565e75bdd6356c3c4de9d14a8"
server_key_version: "encryptionkey@0"
I20260808 22:31:35.308885 616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:35.319130 616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:35.321491 616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:35.321642 616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:35.322121 616 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:31:35.322836 616 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:31:35.322882 616 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:35.322928 616 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:31:35.322958 616 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:35.332289 616 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.129:46723
I20260808 22:31:35.332341 808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.129:46723 every 8 connection(s)
I20260808 22:31:35.333173 616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
I20260808 22:31:35.335742 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 616
I20260808 22:31:35.335851 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/ts-0/wal/instance
I20260808 22:31:35.336114 32094 external_mini_cluster.cc:1428] Setting key 0720eda4e700968e5289350a742d3afa
Aug 08 22:31:35 dist-test-slave-266d krb5kdc[439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228295, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.129@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.87.130@KRBTEST.COM; defaulting to no policy
I20260808 22:31:35.347301 575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:35.334859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:34653 (local address 127.31.87.190:40785)
0808 22:31:35.335025 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:35.335030 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:35.335853 (+ 823us) server_negotiation.cc:407] Connection header received
0808 22:31:35.337511 (+ 1658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:35.337513 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:35.337556 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:35.337632 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:35.338788 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:35.339515 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:35.341135 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:35.341348 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:35.341421 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:35.344056 (+ 2635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:35.344059 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:35.344061 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:35.344085 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:35.345387 (+ 1302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:35.345867 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:35.345870 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:35.345871 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:35.345944 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:35.346251 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:35.346254 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:35.346255 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:35.346449 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:35.346534 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:35.347003 (+ 469us) server_negotiation.cc:299] Negotiation successful
0808 22:31:35.347137 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 22:31:35.348032 811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:35.335244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:40785 (local address 127.31.87.129:34653)
0808 22:31:35.335711 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:35.335753 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:35.337285 (+ 1532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:35.337803 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:35.337810 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:35.338169 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:35.338630 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:35.338644 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:35.339651 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:35.339655 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:35.340982 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:35.340995 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:35.341498 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:35.341502 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:35.341706 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:35.342289 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:31:35.342311 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:35.343904 (+ 1593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:35.345492 (+ 1588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:35.345498 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:35.345512 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:35.345773 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:35.346024 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:35.346028 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:35.346030 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:35.346162 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:35.346547 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:35.346554 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:35.346784 (+ 230us) client_negotiation.cc:769] Sending connection context
0808 22:31:35.346943 (+ 159us) client_negotiation.cc:240] Negotiation successful
0808 22:31:35.347242 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":112,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.130@KRBTEST.COM" created.
I20260808 22:31:35.349336 809 heartbeater.cc:347] Connected to a master server at 127.31.87.190:40785
I20260808 22:31:35.349566 809 heartbeater.cc:464] Registering TS with master...
I20260808 22:31:35.350037 809 heartbeater.cc:511] Master 127.31.87.190:40785 requested a full tablet report, sending...
I20260808 22:31:35.351657 513 ts_manager.cc:194] Registered new tserver with Master: a029a9f6238c462a842b7dd0c133ba7c (127.31.87.129:46723)
I20260808 22:31:35.353308 513 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.129@KRBTEST.COM'} at 127.31.87.129:34653
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab.
Entry for principal kudu/127.31.87.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab.
I20260808 22:31:35.364017 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.130:0
--local_ip_for_outbound_sockets=127.31.87.130
--webserver_interface=127.31.87.130
--webserver_port=0
--tserver_master_addrs=127.31.87.190:40785
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab
--principal=kudu/127.31.87.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:33253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:31:35.469509 816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:35.469735 816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:35.469784 816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:35.473675 816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:35.473722 816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:35.473783 816 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.130
I20260808 22:31:35.478714 816 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:33253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab
--principal=kudu/127.31.87.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.87.130
--webserver_port=0
--tserver_master_addrs=127.31.87.190:40785
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.816
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.130
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:35.479560 816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:35.480419 816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:35.489490 823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:35.489490 825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:35.489689 816 server_base.cc:1034] running on GCE node
W20260808 22:31:35.489791 827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:35.490156 816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:35.490720 816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:35.491863 816 hybrid_clock.cc:648] HybridClock initialized: now 1786228295491839 us; error 37 us; skew 500 ppm
Aug 08 22:31:35 dist-test-slave-266d krb5kdc[439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228295, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:31:35.494374 816 init.cc:377] Logged in from keytab as kudu/127.31.87.130@KRBTEST.COM (short username kudu)
I20260808 22:31:35.495165 816 webserver.cc:460] Webserver started at http://127.31.87.130:37885/ using document root <none> and password file <none>
I20260808 22:31:35.495661 816 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:35.495718 816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:35.495915 816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:35.497838 816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/ts-1/data/instance:
uuid: "9f96e19bc4c2497caa8fe3ba1880840f"
format_stamp: "Formatted at 2026-08-08 22:31:35 on dist-test-slave-266d"
server_key: "8351acdd30e7d21e979e718fe998affd"
server_key_iv: "6967271b93158df816a8a457c25b728a"
server_key_version: "encryptionkey@0"
I20260808 22:31:35.498344 816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/ts-1/wal/instance:
uuid: "9f96e19bc4c2497caa8fe3ba1880840f"
format_stamp: "Formatted at 2026-08-08 22:31:35 on dist-test-slave-266d"
server_key: "8351acdd30e7d21e979e718fe998affd"
server_key_iv: "6967271b93158df816a8a457c25b728a"
server_key_version: "encryptionkey@0"
I20260808 22:31:35.501700 816 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:31:35.503839 838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:35.504894 816 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:31:35.504994 816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/ts-1/wal
uuid: "9f96e19bc4c2497caa8fe3ba1880840f"
format_stamp: "Formatted at 2026-08-08 22:31:35 on dist-test-slave-266d"
server_key: "8351acdd30e7d21e979e718fe998affd"
server_key_iv: "6967271b93158df816a8a457c25b728a"
server_key_version: "encryptionkey@0"
I20260808 22:31:35.505079 816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:35.520740 816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:35.523278 816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:35.523437 816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:35.523958 816 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:31:35.524699 816 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:31:35.524746 816 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:35.524796 816 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:31:35.524822 816 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:35.533296 816 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.130:42751
I20260808 22:31:35.533329 1037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.130:42751 every 8 connection(s)
I20260808 22:31:35.534087 816 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
I20260808 22:31:35.539129 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 816
I20260808 22:31:35.539227 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/ts-1/wal/instance
I20260808 22:31:35.539451 32094 external_mini_cluster.cc:1428] Setting key a97b86f71acdf834bdb45ba5c3b285d7
Aug 08 22:31:35 dist-test-slave-266d krb5kdc[439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228295, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.130@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:31:35.547075 575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:35.535658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:40279 (local address 127.31.87.190:40785)
0808 22:31:35.535774 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:35.535778 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:35.536539 (+ 761us) server_negotiation.cc:407] Connection header received
0808 22:31:35.537798 (+ 1259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:35.537801 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:35.537851 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:35.537914 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:35.538965 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:35.539575 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:35.540865 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:35.541083 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:35.541200 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:35.543718 (+ 2518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:35.543722 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:35.543724 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:35.543747 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:35.545017 (+ 1270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:35.545634 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:35.545637 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:35.545639 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:35.545718 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:35.546056 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:35.546058 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:35.546059 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:35.546240 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:35.546330 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:35.546781 (+ 451us) server_negotiation.cc:299] Negotiation successful
0808 22:31:35.546906 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 22:31:35.547708 1046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:35.535960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:40785 (local address 127.31.87.130:40279)
0808 22:31:35.536393 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:35.536433 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:35.537591 (+ 1158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:35.538057 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:35.538064 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:35.538398 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:35.538831 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:35.538844 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:35.539721 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:35.539725 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:35.540745 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:35.540758 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:35.541258 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:35.541263 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:35.541439 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:35.541968 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:31:35.541988 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:35.543550 (+ 1562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:35.545164 (+ 1614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:35.545170 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:35.545185 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:35.545526 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:35.545802 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:35.545805 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:35.545807 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:35.545976 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:35.546341 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:35.546345 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:35.546549 (+ 204us) client_negotiation.cc:769] Sending connection context
0808 22:31:35.546703 (+ 154us) client_negotiation.cc:240] Negotiation successful
0808 22:31:35.546928 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":121,"threads_started":1}
I20260808 22:31:35.548691 1038 heartbeater.cc:347] Connected to a master server at 127.31.87.190:40785
I20260808 22:31:35.548897 1038 heartbeater.cc:464] Registering TS with master...
I20260808 22:31:35.549360 1038 heartbeater.cc:511] Master 127.31.87.190:40785 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.31.87.131@KRBTEST.COM; defaulting to no policy
I20260808 22:31:35.550338 513 ts_manager.cc:194] Registered new tserver with Master: 9f96e19bc4c2497caa8fe3ba1880840f (127.31.87.130:42751)
I20260808 22:31:35.550925 513 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.130@KRBTEST.COM'} at 127.31.87.130:40279
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab.
Entry for principal kudu/127.31.87.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab.
I20260808 22:31:35.565692 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.131:0
--local_ip_for_outbound_sockets=127.31.87.131
--webserver_interface=127.31.87.131
--webserver_port=0
--tserver_master_addrs=127.31.87.190:40785
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab
--principal=kudu/127.31.87.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:33253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:31:35.673583 1050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:35.673864 1050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:35.673918 1050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:35.677870 1050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:35.677920 1050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:35.677990 1050 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.131
I20260808 22:31:35.683086 1050 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:33253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab
--principal=kudu/127.31.87.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.87.131
--webserver_port=0
--tserver_master_addrs=127.31.87.190:40785
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1050
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.131
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:35.684031 1050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:35.684952 1050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:35.694273 1060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:35.694420 1057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:35.694348 1056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:35.694329 1050 server_base.cc:1034] running on GCE node
I20260808 22:31:35.694875 1050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:35.695482 1050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:35.696638 1050 hybrid_clock.cc:648] HybridClock initialized: now 1786228295696621 us; error 34 us; skew 500 ppm
Aug 08 22:31:35 dist-test-slave-266d krb5kdc[439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228295, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:31:35.699182 1050 init.cc:377] Logged in from keytab as kudu/127.31.87.131@KRBTEST.COM (short username kudu)
I20260808 22:31:35.699965 1050 webserver.cc:460] Webserver started at http://127.31.87.131:45275/ using document root <none> and password file <none>
I20260808 22:31:35.700479 1050 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:35.700541 1050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:35.700810 1050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:35.702745 1050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/ts-2/data/instance:
uuid: "081f2654463c4a818aad3e57ae855a10"
format_stamp: "Formatted at 2026-08-08 22:31:35 on dist-test-slave-266d"
server_key: "c51fdb4c0ad27ce5bee013491175f3d2"
server_key_iv: "6847899179698599939c78345d49efe9"
server_key_version: "encryptionkey@0"
I20260808 22:31:35.703270 1050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/ts-2/wal/instance:
uuid: "081f2654463c4a818aad3e57ae855a10"
format_stamp: "Formatted at 2026-08-08 22:31:35 on dist-test-slave-266d"
server_key: "c51fdb4c0ad27ce5bee013491175f3d2"
server_key_iv: "6847899179698599939c78345d49efe9"
server_key_version: "encryptionkey@0"
I20260808 22:31:35.706705 1050 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 22:31:35.708941 1075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:35.710052 1050 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:31:35.710170 1050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/ts-2/wal
uuid: "081f2654463c4a818aad3e57ae855a10"
format_stamp: "Formatted at 2026-08-08 22:31:35 on dist-test-slave-266d"
server_key: "c51fdb4c0ad27ce5bee013491175f3d2"
server_key_iv: "6847899179698599939c78345d49efe9"
server_key_version: "encryptionkey@0"
I20260808 22:31:35.710256 1050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:35.726346 1050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:35.728696 1050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:35.728853 1050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:35.729408 1050 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:31:35.730182 1050 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:31:35.730229 1050 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:35.730319 1050 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:31:35.730347 1050 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:35.741393 1050 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.131:40395
I20260808 22:31:35.741443 1250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.131:40395 every 8 connection(s)
I20260808 22:31:35.742197 1050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
Aug 08 22:31:35 dist-test-slave-266d krb5kdc[439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228295, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.131@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:31:35.751312 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 1050
I20260808 22:31:35.751427 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786228291013016-32094-0/minicluster-data/ts-2/wal/instance
I20260808 22:31:35.751695 32094 external_mini_cluster.cc:1428] Setting key ef35f16620f856cf94ca39633b5fd9f8
I20260808 22:31:35.754519 575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:35.743837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:45809 (local address 127.31.87.190:40785)
0808 22:31:35.743967 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:35.743971 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:35.744716 (+ 745us) server_negotiation.cc:407] Connection header received
0808 22:31:35.746000 (+ 1284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:35.746003 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:35.746050 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:35.746121 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:35.747117 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:35.747630 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:35.748636 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:35.748838 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:35.748905 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:35.751029 (+ 2124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:35.751032 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:35.751034 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:35.751060 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:35.752593 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:35.753161 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:35.753165 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:35.753167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:35.753245 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:35.753588 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:35.753591 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:35.753592 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:35.753826 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:35.753902 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:35.754314 (+ 412us) server_negotiation.cc:299] Negotiation successful
0808 22:31:35.754405 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 22:31:35.755228 1256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:35.744166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:40785 (local address 127.31.87.131:45809)
0808 22:31:35.744570 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:35.744612 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:35.745818 (+ 1206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:35.746260 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:35.746267 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:35.746585 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:35.747008 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:35.747020 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:35.747715 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:35.747717 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:35.748548 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:35.748557 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:35.748940 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:35.748942 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:35.749069 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:35.749551 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:31:35.749566 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:35.750866 (+ 1300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:35.752712 (+ 1846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:35.752716 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:35.752727 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:35.753054 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:35.753328 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:35.753331 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:35.753333 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:35.753501 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:35.753917 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:35.753922 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:35.754130 (+ 208us) client_negotiation.cc:769] Sending connection context
0808 22:31:35.754272 (+ 142us) client_negotiation.cc:240] Negotiation successful
0808 22:31:35.754500 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":105,"threads_started":1}
I20260808 22:31:35.756183 1253 heartbeater.cc:347] Connected to a master server at 127.31.87.190:40785
I20260808 22:31:35.756407 1253 heartbeater.cc:464] Registering TS with master...
I20260808 22:31:35.756856 1253 heartbeater.cc:511] Master 127.31.87.190:40785 requested a full tablet report, sending...
I20260808 22:31:35.757774 513 ts_manager.cc:194] Registered new tserver with Master: 081f2654463c4a818aad3e57ae855a10 (127.31.87.131:40395)
I20260808 22:31:35.758291 513 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.131@KRBTEST.COM'} at 127.31.87.131:45809
I20260808 22:31:35.765192 32094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:31:35.775676 575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:35.769209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51452 (local address 127.31.87.190:40785)
0808 22:31:35.769323 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:35.769327 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:35.769508 (+ 181us) server_negotiation.cc:407] Connection header received
0808 22:31:35.769595 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:35.769597 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:35.769638 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:35.769722 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:35.770295 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:35.770797 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:35.771527 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:35.771750 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:35.771828 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:35.772947 (+ 1119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:35.772950 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:35.772952 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:35.772971 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:35.774377 (+ 1406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:35.774754 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:35.774756 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:35.774757 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:35.774828 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:35.775007 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:35.775009 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:35.775009 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:35.775195 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:35.775291 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:35.775445 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 22:31:35.775528 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 22:31:35.781064 513 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51452:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:31:35.784128 513 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:31:35.798853 1272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:35.793560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.131:40395 (local address 127.0.0.1:39508)
0808 22:31:35.793886 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:35.793937 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:35.794184 (+ 247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:35.794999 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:35.795004 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:35.795025 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:31:35.795134 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:35.795141 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:35.796435 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:35.796438 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:35.797490 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:35.797500 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:35.798313 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:35.798317 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:35.798430 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:35.798470 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 22:31:35.798583 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 22:31:35.798709 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":77,"threads_started":1}
I20260808 22:31:35.798866 1273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:35.793739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39508 (local address 127.31.87.131:40395)
0808 22:31:35.794048 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:35.794061 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:35.794088 (+ 27us) server_negotiation.cc:407] Connection header received
0808 22:31:35.794394 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:35.794401 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:35.794636 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:35.794809 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:31:35.795270 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:35.796326 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:35.797644 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:35.798195 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:35.798295 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:35.798508 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:35.798621 (+ 113us) server_negotiation.cc:299] Negotiation successful
0808 22:31:35.798712 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":100,"threads_started":1}
I20260808 22:31:35.800414 1275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:35.794240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.130:42751 (local address 127.0.0.1:48296)
0808 22:31:35.794917 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:35.794930 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:35.795019 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:35.795802 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:35.795804 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:35.795817 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:31:35.796014 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:35.796026 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:35.797496 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:35.797499 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:35.798681 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:35.798691 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:35.800000 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:35.800005 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:35.800109 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:35.800136 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 22:31:35.800211 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 22:31:35.800281 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"mutex_wait_us":56,"thread_start_us":89,"threads_started":1}
I20260808 22:31:35.800576 1279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:35.794551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:46723 (local address 127.0.0.1:42492)
0808 22:31:35.795204 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:35.795216 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:35.795308 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:35.795786 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:35.795789 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:35.795806 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:31:35.795981 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:35.795989 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:35.797659 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:35.797662 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:35.798747 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:35.798754 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:35.800168 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:35.800171 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:35.800277 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:35.800302 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 22:31:35.800373 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 22:31:35.800443 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"mutex_wait_us":27,"spinlock_wait_cycles":1280,"thread_start_us":86,"threads_started":1}
I20260808 22:31:35.800850 1278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:35.794425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48296 (local address 127.31.87.130:42751)
0808 22:31:35.795435 (+ 1010us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:35.795448 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:35.795465 (+ 17us) server_negotiation.cc:407] Connection header received
0808 22:31:35.795523 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:35.795526 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:35.795662 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:35.796025 (+ 363us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:31:35.796187 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:35.797332 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:35.798846 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:35.799861 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:35.799963 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:35.800090 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:35.800603 (+ 513us) server_negotiation.cc:299] Negotiation successful
0808 22:31:35.800699 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":910,"thread_start_us":161,"threads_started":1}
I20260808 22:31:35.800974 1281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:35.794968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42492 (local address 127.31.87.129:46723)
0808 22:31:35.795332 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:35.795343 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:35.795365 (+ 22us) server_negotiation.cc:407] Connection header received
0808 22:31:35.795436 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:35.795441 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:35.795636 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:35.795784 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:31:35.796110 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:35.797530 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:35.798860 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:35.799861 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:35.799963 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:35.800090 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:35.800727 (+ 637us) server_negotiation.cc:299] Negotiation successful
0808 22:31:35.800828 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":145,"threads_started":1}
I20260808 22:31:35.802381 1150 tablet_service.cc:1467] Processing CreateTablet for tablet 940ab7a78bb843a3b21adad160db1f4b (DEFAULT_TABLE table=test-table [id=1017f7ed20764fdfae7dfd6bfea239cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:31:35.803033 712 tablet_service.cc:1467] Processing CreateTablet for tablet 940ab7a78bb843a3b21adad160db1f4b (DEFAULT_TABLE table=test-table [id=1017f7ed20764fdfae7dfd6bfea239cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:31:35.803443 1150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 940ab7a78bb843a3b21adad160db1f4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:35.803830 712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 940ab7a78bb843a3b21adad160db1f4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:35.806066 920 tablet_service.cc:1467] Processing CreateTablet for tablet 940ab7a78bb843a3b21adad160db1f4b (DEFAULT_TABLE table=test-table [id=1017f7ed20764fdfae7dfd6bfea239cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:31:35.807062 920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 940ab7a78bb843a3b21adad160db1f4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:35.812641 1282 tablet_bootstrap.cc:492] T 940ab7a78bb843a3b21adad160db1f4b P 081f2654463c4a818aad3e57ae855a10: Bootstrap starting.
I20260808 22:31:35.813259 1284 tablet_bootstrap.cc:492] T 940ab7a78bb843a3b21adad160db1f4b P a029a9f6238c462a842b7dd0c133ba7c: Bootstrap starting.
I20260808 22:31:35.814005 1285 tablet_bootstrap.cc:492] T 940ab7a78bb843a3b21adad160db1f4b P 9f96e19bc4c2497caa8fe3ba1880840f: Bootstrap starting.
I20260808 22:31:35.815001 1282 tablet_bootstrap.cc:654] T 940ab7a78bb843a3b21adad160db1f4b P 081f2654463c4a818aad3e57ae855a10: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:35.815714 1284 tablet_bootstrap.cc:654] T 940ab7a78bb843a3b21adad160db1f4b P a029a9f6238c462a842b7dd0c133ba7c: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:35.816179 1285 tablet_bootstrap.cc:654] T 940ab7a78bb843a3b21adad160db1f4b P 9f96e19bc4c2497caa8fe3ba1880840f: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:35.816612 1282 log.cc:824] T 940ab7a78bb843a3b21adad160db1f4b P 081f2654463c4a818aad3e57ae855a10: Log is configured to *not* fsync() on all Append() calls
I20260808 22:31:35.816852 1285 log.cc:824] T 940ab7a78bb843a3b21adad160db1f4b P 9f96e19bc4c2497caa8fe3ba1880840f: Log is configured to *not* fsync() on all Append() calls
I20260808 22:31:35.816977 1284 log.cc:824] T 940ab7a78bb843a3b21adad160db1f4b P a029a9f6238c462a842b7dd0c133ba7c: Log is configured to *not* fsync() on all Append() calls
I20260808 22:31:35.818810 1284 tablet_bootstrap.cc:492] T 940ab7a78bb843a3b21adad160db1f4b P a029a9f6238c462a842b7dd0c133ba7c: No bootstrap required, opened a new log
I20260808 22:31:35.818831 1285 tablet_bootstrap.cc:492] T 940ab7a78bb843a3b21adad160db1f4b P 9f96e19bc4c2497caa8fe3ba1880840f: No bootstrap required, opened a new log
I20260808 22:31:35.818876 1282 tablet_bootstrap.cc:492] T 940ab7a78bb843a3b21adad160db1f4b P 081f2654463c4a818aad3e57ae855a10: No bootstrap required, opened a new log
I20260808 22:31:35.819012 1285 ts_tablet_manager.cc:1397] T 940ab7a78bb843a3b21adad160db1f4b P 9f96e19bc4c2497caa8fe3ba1880840f: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 22:31:35.819012 1282 ts_tablet_manager.cc:1397] T 940ab7a78bb843a3b21adad160db1f4b P 081f2654463c4a818aad3e57ae855a10: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 22:31:35.819048 1284 ts_tablet_manager.cc:1397] T 940ab7a78bb843a3b21adad160db1f4b P a029a9f6238c462a842b7dd0c133ba7c: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 22:31:35.821916 1284 raft_consensus.cc:348] T 940ab7a78bb843a3b21adad160db1f4b P a029a9f6238c462a842b7dd0c133ba7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "081f2654463c4a818aad3e57ae855a10" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 40395 } } peers { permanent_uuid: "9f96e19bc4c2497caa8fe3ba1880840f" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 42751 } } peers { permanent_uuid: "a029a9f6238c462a842b7dd0c133ba7c" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 46723 } }
I20260808 22:31:35.822137 1284 raft_consensus.cc:374] T 940ab7a78bb843a3b21adad160db1f4b P a029a9f6238c462a842b7dd0c133ba7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:35.822211 1284 raft_consensus.cc:729] T 940ab7a78bb843a3b21adad160db1f4b P a029a9f6238c462a842b7dd0c133ba7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a029a9f6238c462a842b7dd0c133ba7c, State: Initialized, Role: FOLLOWER
I20260808 22:31:35.822634 1284 consensus_queue.cc:260] T 940ab7a78bb843a3b21adad160db1f4b P a029a9f6238c462a842b7dd0c133ba7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "081f2654463c4a818aad3e57ae855a10" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 40395 } } peers { permanent_uuid: "9f96e19bc4c2497caa8fe3ba1880840f" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 42751 } } peers { permanent_uuid: "a029a9f6238c462a842b7dd0c133ba7c" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 46723 } }
I20260808 22:31:35.822798 1282 raft_consensus.cc:348] T 940ab7a78bb843a3b21adad160db1f4b P 081f2654463c4a818aad3e57ae855a10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "081f2654463c4a818aad3e57ae855a10" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 40395 } } peers { permanent_uuid: "9f96e19bc4c2497caa8fe3ba1880840f" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 42751 } } peers { permanent_uuid: "a029a9f6238c462a842b7dd0c133ba7c" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 46723 } }
I20260808 22:31:35.822798 1285 raft_consensus.cc:348] T 940ab7a78bb843a3b21adad160db1f4b P 9f96e19bc4c2497caa8fe3ba1880840f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "081f2654463c4a818aad3e57ae855a10" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 40395 } } peers { permanent_uuid: "9f96e19bc4c2497caa8fe3ba1880840f" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 42751 } } peers { permanent_uuid: "a029a9f6238c462a842b7dd0c133ba7c" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 46723 } }
I20260808 22:31:35.823057 1285 raft_consensus.cc:374] T 940ab7a78bb843a3b21adad160db1f4b P 9f96e19bc4c2497caa8fe3ba1880840f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:35.823135 1282 raft_consensus.cc:374] T 940ab7a78bb843a3b21adad160db1f4b P 081f2654463c4a818aad3e57ae855a10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:35.823163 1285 raft_consensus.cc:729] T 940ab7a78bb843a3b21adad160db1f4b P 9f96e19bc4c2497caa8fe3ba1880840f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f96e19bc4c2497caa8fe3ba1880840f, State: Initialized, Role: FOLLOWER
I20260808 22:31:35.823227 1282 raft_consensus.cc:729] T 940ab7a78bb843a3b21adad160db1f4b P 081f2654463c4a818aad3e57ae855a10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 081f2654463c4a818aad3e57ae855a10, State: Initialized, Role: FOLLOWER
I20260808 22:31:35.823340 809 heartbeater.cc:503] Master 127.31.87.190:40785 was elected leader, sending a full tablet report...
I20260808 22:31:35.823618 1284 ts_tablet_manager.cc:1428] T 940ab7a78bb843a3b21adad160db1f4b P a029a9f6238c462a842b7dd0c133ba7c: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260808 22:31:35.823654 1282 consensus_queue.cc:260] T 940ab7a78bb843a3b21adad160db1f4b P 081f2654463c4a818aad3e57ae855a10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "081f2654463c4a818aad3e57ae855a10" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 40395 } } peers { permanent_uuid: "9f96e19bc4c2497caa8fe3ba1880840f" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 42751 } } peers { permanent_uuid: "a029a9f6238c462a842b7dd0c133ba7c" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 46723 } }
I20260808 22:31:35.823654 1285 consensus_queue.cc:260] T 940ab7a78bb843a3b21adad160db1f4b P 9f96e19bc4c2497caa8fe3ba1880840f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "081f2654463c4a818aad3e57ae855a10" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 40395 } } peers { permanent_uuid: "9f96e19bc4c2497caa8fe3ba1880840f" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 42751 } } peers { permanent_uuid: "a029a9f6238c462a842b7dd0c133ba7c" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 46723 } }
I20260808 22:31:35.824590 1285 ts_tablet_manager.cc:1428] T 940ab7a78bb843a3b21adad160db1f4b P 9f96e19bc4c2497caa8fe3ba1880840f: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 22:31:35.824580 1282 ts_tablet_manager.cc:1428] T 940ab7a78bb843a3b21adad160db1f4b P 081f2654463c4a818aad3e57ae855a10: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 22:31:35.824931 1253 heartbeater.cc:503] Master 127.31.87.190:40785 was elected leader, sending a full tablet report...
I20260808 22:31:35.825258 1038 heartbeater.cc:503] Master 127.31.87.190:40785 was elected leader, sending a full tablet report...
W20260808 22:31:35.834156 810 tablet.cc:2369] T 940ab7a78bb843a3b21adad160db1f4b P a029a9f6238c462a842b7dd0c133ba7c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 22:31:35.993280 1254 tablet.cc:2369] T 940ab7a78bb843a3b21adad160db1f4b P 081f2654463c4a818aad3e57ae855a10: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:31:36.001122 1293 raft_consensus.cc:482] T 940ab7a78bb843a3b21adad160db1f4b P 081f2654463c4a818aad3e57ae855a10 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:31:36.001314 1293 raft_consensus.cc:504] T 940ab7a78bb843a3b21adad160db1f4b P 081f2654463c4a818aad3e57ae855a10 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "081f2654463c4a818aad3e57ae855a10" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 40395 } } peers { permanent_uuid: "9f96e19bc4c2497caa8fe3ba1880840f" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 42751 } } peers { permanent_uuid: "a029a9f6238c462a842b7dd0c133ba7c" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 46723 } }
I20260808 22:31:36.002449 1293 leader_election.cc:288] T 940ab7a78bb843a3b21adad160db1f4b P 081f2654463c4a818aad3e57ae855a10 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f96e19bc4c2497caa8fe3ba1880840f (127.31.87.130:42751), a029a9f6238c462a842b7dd0c133ba7c (127.31.87.129:46723)
I20260808 22:31:36.006322 1278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:36.002828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:38489 (local address 127.31.87.130:42751)
0808 22:31:36.002958 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:36.002963 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:36.002979 (+ 16us) server_negotiation.cc:407] Connection header received
0808 22:31:36.003030 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:36.003033 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:36.003074 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:36.003139 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:31:36.003456 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:36.003970 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:36.005231 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:36.005843 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:36.005956 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:36.006005 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:36.006162 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 22:31:36.006217 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 22:31:36.006363 1256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:36.002691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.130:42751 (local address 127.31.87.131:38489)
0808 22:31:36.002819 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:36.002831 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:36.002925 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:36.003189 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:36.003192 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:36.003215 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:31:36.003336 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:36.003344 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:36.004063 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:36.004067 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:36.005079 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:36.005118 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:36.005940 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:36.005942 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:36.006052 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:36.006077 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 22:31:36.006151 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 22:31:36.006219 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 22:31:36.006984 1281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:36.003072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:39439 (local address 127.31.87.129:46723)
0808 22:31:36.003220 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:36.003224 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:36.003469 (+ 245us) server_negotiation.cc:407] Connection header received
0808 22:31:36.003592 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:36.003594 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:36.003637 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:36.003695 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:31:36.003941 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:36.004644 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:36.005938 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:36.006470 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:36.006582 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:36.006628 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:36.006822 (+ 194us) server_negotiation.cc:299] Negotiation successful
0808 22:31:36.006881 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 22:31:36.006984 1295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:36.002966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:46723 (local address 127.31.87.131:39439)
0808 22:31:36.003398 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:36.003411 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:36.003497 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:36.003714 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:36.003716 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:36.003734 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:31:36.003843 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:36.003848 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:36.004750 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:36.004752 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:36.005817 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:36.005830 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:36.006642 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:36.006646 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:36.006718 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:36.006734 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 22:31:36.006797 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 22:31:36.006859 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":143,"threads_started":1}
I20260808 22:31:36.007072 959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "940ab7a78bb843a3b21adad160db1f4b" candidate_uuid: "081f2654463c4a818aad3e57ae855a10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f96e19bc4c2497caa8fe3ba1880840f" is_pre_election: true
I20260808 22:31:36.007489 741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "940ab7a78bb843a3b21adad160db1f4b" candidate_uuid: "081f2654463c4a818aad3e57ae855a10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a029a9f6238c462a842b7dd0c133ba7c" is_pre_election: true
I20260808 22:31:36.007509 959 raft_consensus.cc:2454] T 940ab7a78bb843a3b21adad160db1f4b P 9f96e19bc4c2497caa8fe3ba1880840f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 081f2654463c4a818aad3e57ae855a10 in term 0.
I20260808 22:31:36.007735 741 raft_consensus.cc:2454] T 940ab7a78bb843a3b21adad160db1f4b P a029a9f6238c462a842b7dd0c133ba7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 081f2654463c4a818aad3e57ae855a10 in term 0.
I20260808 22:31:36.008072 1076 leader_election.cc:302] T 940ab7a78bb843a3b21adad160db1f4b P 081f2654463c4a818aad3e57ae855a10 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 081f2654463c4a818aad3e57ae855a10, 9f96e19bc4c2497caa8fe3ba1880840f; no voters:
I20260808 22:31:36.008328 1293 raft_consensus.cc:2781] T 940ab7a78bb843a3b21adad160db1f4b P 081f2654463c4a818aad3e57ae855a10 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:31:36.008419 1293 raft_consensus.cc:482] T 940ab7a78bb843a3b21adad160db1f4b P 081f2654463c4a818aad3e57ae855a10 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:31:36.008469 1293 raft_consensus.cc:3037] T 940ab7a78bb843a3b21adad160db1f4b P 081f2654463c4a818aad3e57ae855a10 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:36.009539 1293 raft_consensus.cc:504] T 940ab7a78bb843a3b21adad160db1f4b P 081f2654463c4a818aad3e57ae855a10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "081f2654463c4a818aad3e57ae855a10" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 40395 } } peers { permanent_uuid: "9f96e19bc4c2497caa8fe3ba1880840f" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 42751 } } peers { permanent_uuid: "a029a9f6238c462a842b7dd0c133ba7c" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 46723 } }
I20260808 22:31:36.009845 1293 leader_election.cc:288] T 940ab7a78bb843a3b21adad160db1f4b P 081f2654463c4a818aad3e57ae855a10 [CANDIDATE]: Term 1 election: Requested vote from peers 9f96e19bc4c2497caa8fe3ba1880840f (127.31.87.130:42751), a029a9f6238c462a842b7dd0c133ba7c (127.31.87.129:46723)
I20260808 22:31:36.010150 959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "940ab7a78bb843a3b21adad160db1f4b" candidate_uuid: "081f2654463c4a818aad3e57ae855a10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f96e19bc4c2497caa8fe3ba1880840f"
I20260808 22:31:36.010241 741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "940ab7a78bb843a3b21adad160db1f4b" candidate_uuid: "081f2654463c4a818aad3e57ae855a10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a029a9f6238c462a842b7dd0c133ba7c"
I20260808 22:31:36.010303 959 raft_consensus.cc:3037] T 940ab7a78bb843a3b21adad160db1f4b P 9f96e19bc4c2497caa8fe3ba1880840f [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:36.010352 741 raft_consensus.cc:3037] T 940ab7a78bb843a3b21adad160db1f4b P a029a9f6238c462a842b7dd0c133ba7c [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:36.011521 959 raft_consensus.cc:2454] T 940ab7a78bb843a3b21adad160db1f4b P 9f96e19bc4c2497caa8fe3ba1880840f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 081f2654463c4a818aad3e57ae855a10 in term 1.
I20260808 22:31:36.011521 741 raft_consensus.cc:2454] T 940ab7a78bb843a3b21adad160db1f4b P a029a9f6238c462a842b7dd0c133ba7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 081f2654463c4a818aad3e57ae855a10 in term 1.
I20260808 22:31:36.011842 1076 leader_election.cc:302] T 940ab7a78bb843a3b21adad160db1f4b P 081f2654463c4a818aad3e57ae855a10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 081f2654463c4a818aad3e57ae855a10, 9f96e19bc4c2497caa8fe3ba1880840f; no voters:
I20260808 22:31:36.012012 1293 raft_consensus.cc:2781] T 940ab7a78bb843a3b21adad160db1f4b P 081f2654463c4a818aad3e57ae855a10 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:31:36.012231 1293 raft_consensus.cc:686] T 940ab7a78bb843a3b21adad160db1f4b P 081f2654463c4a818aad3e57ae855a10 [term 1 LEADER]: Becoming Leader. State: Replica: 081f2654463c4a818aad3e57ae855a10, State: Running, Role: LEADER
I20260808 22:31:36.012526 1293 consensus_queue.cc:237] T 940ab7a78bb843a3b21adad160db1f4b P 081f2654463c4a818aad3e57ae855a10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "081f2654463c4a818aad3e57ae855a10" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 40395 } } peers { permanent_uuid: "9f96e19bc4c2497caa8fe3ba1880840f" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 42751 } } peers { permanent_uuid: "a029a9f6238c462a842b7dd0c133ba7c" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 46723 } }
I20260808 22:31:36.015772 513 catalog_manager.cc:5495] T 940ab7a78bb843a3b21adad160db1f4b P 081f2654463c4a818aad3e57ae855a10 reported cstate change: term changed from 0 to 1, leader changed from <none> to 081f2654463c4a818aad3e57ae855a10 (127.31.87.131). New cstate: current_term: 1 leader_uuid: "081f2654463c4a818aad3e57ae855a10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "081f2654463c4a818aad3e57ae855a10" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 40395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9f96e19bc4c2497caa8fe3ba1880840f" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 42751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a029a9f6238c462a842b7dd0c133ba7c" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 46723 } health_report { overall_health: UNKNOWN } } }
Aug 08 22:31:36 dist-test-slave-266d krb5kdc[439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.87.129@KRBTEST.COM
I20260808 22:31:36.033957 1281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:36.025930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42506 (local address 127.31.87.129:46723)
0808 22:31:36.026050 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:36.026054 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:36.026231 (+ 177us) server_negotiation.cc:407] Connection header received
0808 22:31:36.026345 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:36.026349 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:36.026389 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:36.026454 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:36.027111 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:36.027569 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:36.028377 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:36.028582 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:36.028674 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:36.031053 (+ 2379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:36.031064 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:36.031077 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:36.031099 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:36.032522 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:36.032892 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:36.032896 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:36.032897 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:36.032979 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:36.033228 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:36.033230 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:36.033231 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:36.033445 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:36.033592 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:36.033724 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 22:31:36.033813 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260808 22:31:36.035164 1045 tablet.cc:2369] T 940ab7a78bb843a3b21adad160db1f4b P 9f96e19bc4c2497caa8fe3ba1880840f: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 22:31:36 dist-test-slave-266d krb5kdc[439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228296, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 22:31:36 dist-test-slave-266d krb5kdc[439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228296, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.87.129@KRBTEST.COM
I20260808 22:31:36.059576 1281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:36.052223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42516 (local address 127.31.87.129:46723)
0808 22:31:36.052336 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:36.052340 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:36.052480 (+ 140us) server_negotiation.cc:407] Connection header received
0808 22:31:36.052594 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:36.052597 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:36.052634 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:36.052694 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:36.053333 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:36.053817 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:36.054592 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:36.054804 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:36.054888 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:36.056965 (+ 2077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:36.056968 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:36.056970 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:36.056992 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:36.058141 (+ 1149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:36.058510 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:36.058513 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:36.058515 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:36.058587 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:36.058818 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:36.058821 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:36.058823 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:36.058969 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:36.059079 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:36.059328 (+ 249us) server_negotiation.cc:299] Negotiation successful
0808 22:31:36.059434 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
Aug 08 22:31:36 dist-test-slave-266d krb5kdc[439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 22:31:36 dist-test-slave-266d krb5kdc[439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.87.129@KRBTEST.COM
I20260808 22:31:36.083312 1281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:36.076042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42526 (local address 127.31.87.129:46723)
0808 22:31:36.076187 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:36.076191 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:36.076326 (+ 135us) server_negotiation.cc:407] Connection header received
0808 22:31:36.076446 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:36.076448 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:36.076487 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:36.076551 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:36.077307 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:36.077781 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:36.078526 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:36.078720 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:36.078785 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:36.080877 (+ 2092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:36.080880 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:36.080882 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:36.080904 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:36.082151 (+ 1247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:36.082478 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:36.082481 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:36.082482 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:36.082543 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:36.082715 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:36.082717 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:36.082718 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:36.082849 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:36.082936 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:36.083075 (+ 139us) server_negotiation.cc:299] Negotiation successful
0808 22:31:36.083166 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260808 22:31:36.084766 679 tablet_service.cc:2908] Rejecting scan request for tablet 940ab7a78bb843a3b21adad160db1f4b: Uninitialized: clean time has not yet been initialized
I20260808 22:31:36.087307 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 616
I20260808 22:31:36.100252 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 816
I20260808 22:31:36.125696 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 1050
I20260808 22:31:36.142068 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 457
2026-08-08T22:31:36Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2505 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:31:36 dist-test-slave-266d krb5kdc[1318](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:31:36 dist-test-slave-266d krb5kdc[1318](info): set up 2 sockets
Aug 08 22:31:36 dist-test-slave-266d krb5kdc[1318](info): commencing operation
krb5kdc: starting...
W20260808 22:31:37.291230 32094 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Aug 08 22:31:37 dist-test-slave-266d krb5kdc[1318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:31:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:31:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.87.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
I20260808 22:31:37.401791 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:39551
--webserver_interface=127.31.87.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:39701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:39551
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:31:37.507527 1334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:37.507829 1334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:37.507894 1334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:37.511966 1334 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:31:37.512037 1334 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 22:31:37.512064 1334 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:37.512092 1334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:37.512118 1334 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:31:37.512152 1334 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:31:37.517338 1334 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:39701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:39551
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:39551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.87.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1334
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:37.518335 1334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:37.519325 1334 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:37.527498 1339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:37.527467 1340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:37.527489 1348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:37.527545 1334 server_base.cc:1034] running on GCE node
I20260808 22:31:37.528276 1334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:37.529337 1334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:37.530504 1334 hybrid_clock.cc:648] HybridClock initialized: now 1786228297530480 us; error 33 us; skew 500 ppm
Aug 08 22:31:37 dist-test-slave-266d krb5kdc[1318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228297, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:31:37.533165 1334 init.cc:377] Logged in from keytab as kudu/127.31.87.190@KRBTEST.COM (short username kudu)
I20260808 22:31:37.534070 1334 webserver.cc:460] Webserver started at http://127.31.87.190:34713/ using document root <none> and password file <none>
I20260808 22:31:37.534598 1334 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:37.534672 1334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:37.534885 1334 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:37.536929 1334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/master-0/data/instance:
uuid: "989bdf9e19a44aa3987414387310f4b9"
format_stamp: "Formatted at 2026-08-08 22:31:37 on dist-test-slave-266d"
server_key: "592c4ef7e7ad6e550f847c84b59d6eff"
server_key_iv: "273b5bb4c3ad152fa9691bb93eee06de"
server_key_version: "encryptionkey@0"
I20260808 22:31:37.537546 1334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/master-0/wal/instance:
uuid: "989bdf9e19a44aa3987414387310f4b9"
format_stamp: "Formatted at 2026-08-08 22:31:37 on dist-test-slave-266d"
server_key: "592c4ef7e7ad6e550f847c84b59d6eff"
server_key_iv: "273b5bb4c3ad152fa9691bb93eee06de"
server_key_version: "encryptionkey@0"
I20260808 22:31:37.541182 1334 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:31:37.543424 1357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:37.544543 1334 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:31:37.544661 1334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "989bdf9e19a44aa3987414387310f4b9"
format_stamp: "Formatted at 2026-08-08 22:31:37 on dist-test-slave-266d"
server_key: "592c4ef7e7ad6e550f847c84b59d6eff"
server_key_iv: "273b5bb4c3ad152fa9691bb93eee06de"
server_key_version: "encryptionkey@0"
I20260808 22:31:37.544754 1334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:37.558204 1334 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:37.560609 1334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:37.560772 1334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:37.568291 1334 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.190:39551
I20260808 22:31:37.568331 1467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.190:39551 every 8 connection(s)
I20260808 22:31:37.569172 1334 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:31:37.571758 1468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:37.576593 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 1334
I20260808 22:31:37.576709 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/master-0/wal/instance
I20260808 22:31:37.576938 32094 external_mini_cluster.cc:1428] Setting key 730664ddcd87447f25ae56ae9fb744d5
I20260808 22:31:37.577158 1468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 989bdf9e19a44aa3987414387310f4b9: Bootstrap starting.
I20260808 22:31:37.579473 1468 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 989bdf9e19a44aa3987414387310f4b9: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:37.580160 1468 log.cc:824] T 00000000000000000000000000000000 P 989bdf9e19a44aa3987414387310f4b9: Log is configured to *not* fsync() on all Append() calls
I20260808 22:31:37.582199 1468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 989bdf9e19a44aa3987414387310f4b9: No bootstrap required, opened a new log
Aug 08 22:31:37 dist-test-slave-266d krb5kdc[1318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:31:37.586172 1468 raft_consensus.cc:348] T 00000000000000000000000000000000 P 989bdf9e19a44aa3987414387310f4b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "989bdf9e19a44aa3987414387310f4b9" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 39551 } }
I20260808 22:31:37.586442 1468 raft_consensus.cc:374] T 00000000000000000000000000000000 P 989bdf9e19a44aa3987414387310f4b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:37.586503 1468 raft_consensus.cc:729] T 00000000000000000000000000000000 P 989bdf9e19a44aa3987414387310f4b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 989bdf9e19a44aa3987414387310f4b9, State: Initialized, Role: FOLLOWER
I20260808 22:31:37.586937 1468 consensus_queue.cc:260] T 00000000000000000000000000000000 P 989bdf9e19a44aa3987414387310f4b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "989bdf9e19a44aa3987414387310f4b9" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 39551 } }
I20260808 22:31:37.587100 1468 raft_consensus.cc:388] T 00000000000000000000000000000000 P 989bdf9e19a44aa3987414387310f4b9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:31:37.587167 1468 raft_consensus.cc:482] T 00000000000000000000000000000000 P 989bdf9e19a44aa3987414387310f4b9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:31:37.587260 1468 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 989bdf9e19a44aa3987414387310f4b9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:37.588105 1468 raft_consensus.cc:504] T 00000000000000000000000000000000 P 989bdf9e19a44aa3987414387310f4b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "989bdf9e19a44aa3987414387310f4b9" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 39551 } }
I20260808 22:31:37.588413 1468 leader_election.cc:302] T 00000000000000000000000000000000 P 989bdf9e19a44aa3987414387310f4b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 989bdf9e19a44aa3987414387310f4b9; no voters:
I20260808 22:31:37.588680 1468 leader_election.cc:288] T 00000000000000000000000000000000 P 989bdf9e19a44aa3987414387310f4b9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:31:37.588781 1474 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 989bdf9e19a44aa3987414387310f4b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:31:37.588974 1474 raft_consensus.cc:686] T 00000000000000000000000000000000 P 989bdf9e19a44aa3987414387310f4b9 [term 1 LEADER]: Becoming Leader. State: Replica: 989bdf9e19a44aa3987414387310f4b9, State: Running, Role: LEADER
I20260808 22:31:37.589344 1474 consensus_queue.cc:237] T 00000000000000000000000000000000 P 989bdf9e19a44aa3987414387310f4b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "989bdf9e19a44aa3987414387310f4b9" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 39551 } }
I20260808 22:31:37.589919 1468 sys_catalog.cc:564] T 00000000000000000000000000000000 P 989bdf9e19a44aa3987414387310f4b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:31:37.590442 1479 sys_catalog.cc:455] T 00000000000000000000000000000000 P 989bdf9e19a44aa3987414387310f4b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 989bdf9e19a44aa3987414387310f4b9. Latest consensus state: current_term: 1 leader_uuid: "989bdf9e19a44aa3987414387310f4b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "989bdf9e19a44aa3987414387310f4b9" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 39551 } } }
I20260808 22:31:37.590561 1479 sys_catalog.cc:458] T 00000000000000000000000000000000 P 989bdf9e19a44aa3987414387310f4b9 [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:37.590963 1495 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:31:37.591353 1472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:37.578305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58018 (local address 127.31.87.190:39551)
0808 22:31:37.578725 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:37.578738 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:37.578772 (+ 34us) server_negotiation.cc:407] Connection header received
0808 22:31:37.579370 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:37.579385 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:37.579723 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:37.580017 (+ 294us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:37.580855 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:37.582280 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:37.583103 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:37.583825 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:37.583992 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:37.586547 (+ 2555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:37.586554 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:37.586571 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:37.586599 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:37.588229 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:37.588647 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:37.588653 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:37.588659 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:37.588828 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:37.589160 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:37.589163 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:37.589165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:37.589627 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:37.589746 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:37.590156 (+ 410us) server_negotiation.cc:299] Negotiation successful
0808 22:31:37.590449 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":136,"threads_started":1}
I20260808 22:31:37.592224 1475 sys_catalog.cc:455] T 00000000000000000000000000000000 P 989bdf9e19a44aa3987414387310f4b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "989bdf9e19a44aa3987414387310f4b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "989bdf9e19a44aa3987414387310f4b9" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 39551 } } }
I20260808 22:31:37.592381 1475 sys_catalog.cc:458] T 00000000000000000000000000000000 P 989bdf9e19a44aa3987414387310f4b9 [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:37.594013 1495 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:31:37.599279 1495 catalog_manager.cc:1347] Generated new cluster ID: 9b88523d722747568d9d32780f4eb3b1
I20260808 22:31:37.599339 1495 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:31:37.650355 1495 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:31:37.651309 1495 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:31:37.663475 1495 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 989bdf9e19a44aa3987414387310f4b9: Generated new TSK 0
I20260808 22:31:37.664109 1495 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.87.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab.
Entry for principal kudu/127.31.87.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab.
I20260808 22:31:37.704170 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.129:0
--local_ip_for_outbound_sockets=127.31.87.129
--webserver_interface=127.31.87.129
--webserver_port=0
--tserver_master_addrs=127.31.87.190:39551
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab
--principal=kudu/127.31.87.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:39701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:31:37.828918 1524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:37.829185 1524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:37.829231 1524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:37.833120 1524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:37.833176 1524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:37.833212 1524 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 22:31:37.833248 1524 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.129
I20260808 22:31:37.838243 1524 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:39701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab
--principal=kudu/127.31.87.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.87.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.87.190:39551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.129
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:37.839361 1524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:37.840554 1524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:37.850831 1534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:37.850862 1533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:37.850847 1538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:37.851040 1524 server_base.cc:1034] running on GCE node
I20260808 22:31:37.851401 1524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:37.851975 1524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:37.853135 1524 hybrid_clock.cc:648] HybridClock initialized: now 1786228297853108 us; error 39 us; skew 500 ppm
Aug 08 22:31:37 dist-test-slave-266d krb5kdc[1318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228297, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:31:37.860513 1524 init.cc:377] Logged in from keytab as kudu/127.31.87.129@KRBTEST.COM (short username kudu)
I20260808 22:31:37.861364 1524 webserver.cc:460] Webserver started at http://127.31.87.129:35079/ using document root <none> and password file <none>
I20260808 22:31:37.861871 1524 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:37.861929 1524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:37.862093 1524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:37.864012 1524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-0/data/instance:
uuid: "9f36d03373d54b269ac978d372329799"
format_stamp: "Formatted at 2026-08-08 22:31:37 on dist-test-slave-266d"
server_key: "20867a0842d8856a25fe94dd1b22193c"
server_key_iv: "5b3f18b137ac04c3d24b4d702c1b73b1"
server_key_version: "encryptionkey@0"
I20260808 22:31:37.864528 1524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-0/wal/instance:
uuid: "9f36d03373d54b269ac978d372329799"
format_stamp: "Formatted at 2026-08-08 22:31:37 on dist-test-slave-266d"
server_key: "20867a0842d8856a25fe94dd1b22193c"
server_key_iv: "5b3f18b137ac04c3d24b4d702c1b73b1"
server_key_version: "encryptionkey@0"
I20260808 22:31:37.868055 1524 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 22:31:37.870371 1549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:37.871474 1524 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:31:37.871568 1524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-0/wal
uuid: "9f36d03373d54b269ac978d372329799"
format_stamp: "Formatted at 2026-08-08 22:31:37 on dist-test-slave-266d"
server_key: "20867a0842d8856a25fe94dd1b22193c"
server_key_iv: "5b3f18b137ac04c3d24b4d702c1b73b1"
server_key_version: "encryptionkey@0"
I20260808 22:31:37.871658 1524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:37.885160 1524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:37.887908 1524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:37.888062 1524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:37.889374 1524 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:31:37.889432 1524 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:37.889480 1524 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:31:37.889508 1524 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 22:31:37 dist-test-slave-266d krb5kdc[1318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228297, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.129@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:31:37.899602 1524 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.129:40357
I20260808 22:31:37.899657 1742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.129:40357 every 8 connection(s)
I20260808 22:31:37.900390 1524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
I20260808 22:31:37.901911 1472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:37.890073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:53785 (local address 127.31.87.190:39551)
0808 22:31:37.890255 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:37.890260 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:37.891239 (+ 979us) server_negotiation.cc:407] Connection header received
0808 22:31:37.892162 (+ 923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:37.892166 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:37.892226 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:37.892300 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:37.893492 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:37.894038 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:37.895153 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:37.895460 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:37.895551 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:37.898594 (+ 3043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:37.898598 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:37.898610 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:37.898636 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:37.899989 (+ 1353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:37.900512 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:37.900516 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:37.900518 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:37.900606 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:37.900986 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:37.900990 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:37.900991 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:37.901229 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:37.901306 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:37.901699 (+ 393us) server_negotiation.cc:299] Negotiation successful
0808 22:31:37.901792 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 22:31:37.902590 1560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:37.890646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:39551 (local address 127.31.87.129:53785)
0808 22:31:37.891087 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:37.891148 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:37.891973 (+ 825us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:37.892479 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:37.892487 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:37.892872 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:37.893352 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:37.893365 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:37.894163 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:37.894166 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:37.895041 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:37.895051 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:37.895610 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:37.895615 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:37.895761 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:37.896275 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:31:37.896293 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:37.898443 (+ 2150us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:37.900094 (+ 1651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:37.900100 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:37.900120 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:37.900400 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:37.900711 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:37.900714 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:37.900716 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:37.900874 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:37.901312 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:37.901317 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:37.901514 (+ 197us) client_negotiation.cc:769] Sending connection context
0808 22:31:37.901662 (+ 148us) client_negotiation.cc:240] Negotiation successful
0808 22:31:37.901884 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":126,"threads_started":1}
I20260808 22:31:37.903684 1743 heartbeater.cc:347] Connected to a master server at 127.31.87.190:39551
I20260808 22:31:37.903824 1743 heartbeater.cc:464] Registering TS with master...
I20260808 22:31:37.904356 1743 heartbeater.cc:511] Master 127.31.87.190:39551 requested a full tablet report, sending...
I20260808 22:31:37.905877 1378 ts_manager.cc:194] Registered new tserver with Master: 9f36d03373d54b269ac978d372329799 (127.31.87.129:40357)
I20260808 22:31:37.906939 1378 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.129@KRBTEST.COM'} at 127.31.87.129:53785
I20260808 22:31:37.910507 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 1524
I20260808 22:31:37.910629 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-0/wal/instance
I20260808 22:31:37.910926 32094 external_mini_cluster.cc:1428] Setting key 0aac502268f2af400fd4bef731083316
I20260808 22:31:37.911989 1472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:37.905869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:39187 (local address 127.31.87.190:39551)
0808 22:31:37.905968 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:37.905971 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:37.906114 (+ 143us) server_negotiation.cc:407] Connection header received
0808 22:31:37.906216 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:37.906219 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:37.906259 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:37.906320 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:37.906710 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:37.907181 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:37.908175 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:37.908382 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:37.908459 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:37.909468 (+ 1009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:37.909471 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:37.909473 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:37.909493 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:37.910507 (+ 1014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:37.910921 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:37.910923 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:37.910924 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:37.911000 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:37.911255 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:37.911258 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:37.911260 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:37.911431 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:37.911520 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:37.911800 (+ 280us) server_negotiation.cc:299] Negotiation successful
0808 22:31:37.911884 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 22:31:37.912055 1753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:37.905800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:39551 (local address 127.31.87.129:39187)
0808 22:31:37.906046 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:37.906057 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:37.906148 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:37.906343 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:37.906346 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:37.906516 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:37.906608 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:37.906614 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:37.907296 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:37.907299 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:37.908054 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:37.908065 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:37.908488 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:37.908492 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:37.908580 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:37.908980 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:31:37.908994 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:37.909348 (+ 354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:37.910608 (+ 1260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:37.910612 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:37.910613 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:37.910834 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:37.911094 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:37.911097 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:37.911099 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:37.911179 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:37.911543 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:37.911546 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:37.911647 (+ 101us) client_negotiation.cc:769] Sending connection context
0808 22:31:37.911753 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 22:31:37.911874 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":88,"threads_started":1}
WARNING: no policy specified for kudu/127.31.87.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab.
Entry for principal kudu/127.31.87.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab.
I20260808 22:31:37.935290 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.130:0
--local_ip_for_outbound_sockets=127.31.87.130
--webserver_interface=127.31.87.130
--webserver_port=0
--tserver_master_addrs=127.31.87.190:39551
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab
--principal=kudu/127.31.87.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:39701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:31:38.059320 1756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:38.059562 1756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:38.059603 1756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:38.063549 1756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:38.063602 1756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:38.063639 1756 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 22:31:38.063673 1756 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.130
I20260808 22:31:38.068722 1756 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:39701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab
--principal=kudu/127.31.87.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.87.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.87.190:39551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.130
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:38.069694 1756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:38.070633 1756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:38.079932 1765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:38.080039 1762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:38.079936 1763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:38.080129 1756 server_base.cc:1034] running on GCE node
I20260808 22:31:38.080492 1756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:38.081159 1756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:38.082324 1756 hybrid_clock.cc:648] HybridClock initialized: now 1786228298082305 us; error 32 us; skew 500 ppm
Aug 08 22:31:38 dist-test-slave-266d krb5kdc[1318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228298, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:31:38.088078 1756 init.cc:377] Logged in from keytab as kudu/127.31.87.130@KRBTEST.COM (short username kudu)
I20260808 22:31:38.088912 1756 webserver.cc:460] Webserver started at http://127.31.87.130:43561/ using document root <none> and password file <none>
I20260808 22:31:38.089475 1756 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:38.089542 1756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:38.089766 1756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:38.091946 1756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-1/data/instance:
uuid: "dba5657cda984625a81e9fa4dda06b8b"
format_stamp: "Formatted at 2026-08-08 22:31:38 on dist-test-slave-266d"
server_key: "3d2fbd9b4494ba4734bbb33464fd96d8"
server_key_iv: "47dde4da91bf76ce68390b8ef81e05ae"
server_key_version: "encryptionkey@0"
I20260808 22:31:38.092483 1756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-1/wal/instance:
uuid: "dba5657cda984625a81e9fa4dda06b8b"
format_stamp: "Formatted at 2026-08-08 22:31:38 on dist-test-slave-266d"
server_key: "3d2fbd9b4494ba4734bbb33464fd96d8"
server_key_iv: "47dde4da91bf76ce68390b8ef81e05ae"
server_key_version: "encryptionkey@0"
I20260808 22:31:38.095960 1756 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 22:31:38.098237 1773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:38.099339 1756 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:31:38.099443 1756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-1/wal
uuid: "dba5657cda984625a81e9fa4dda06b8b"
format_stamp: "Formatted at 2026-08-08 22:31:38 on dist-test-slave-266d"
server_key: "3d2fbd9b4494ba4734bbb33464fd96d8"
server_key_iv: "47dde4da91bf76ce68390b8ef81e05ae"
server_key_version: "encryptionkey@0"
I20260808 22:31:38.099529 1756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:38.109390 1756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:38.111825 1756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:38.111984 1756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:38.113339 1756 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:31:38.113389 1756 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:38.113438 1756 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:31:38.113468 1756 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 22:31:38 dist-test-slave-266d krb5kdc[1318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228298, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.130@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:31:38.123507 1756 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.130:44213
I20260808 22:31:38.123588 1982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.130:44213 every 8 connection(s)
I20260808 22:31:38.124352 1756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
I20260808 22:31:38.126616 1472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:38.114026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:56627 (local address 127.31.87.190:39551)
0808 22:31:38.114141 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:38.114145 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:38.115112 (+ 967us) server_negotiation.cc:407] Connection header received
0808 22:31:38.115994 (+ 882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:38.115998 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:38.116057 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:38.116133 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:38.117604 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:38.118405 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:38.120097 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:38.120437 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:38.120549 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:38.123315 (+ 2766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:38.123318 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:38.123320 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:38.123346 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:38.124845 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:38.125427 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:38.125430 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:38.125432 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:38.125512 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:38.125749 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:38.125751 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:38.125753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:38.125920 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:38.125989 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:38.126392 (+ 403us) server_negotiation.cc:299] Negotiation successful
0808 22:31:38.126492 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 22:31:38.127355 1785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:38.114517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:39551 (local address 127.31.87.130:56627)
0808 22:31:38.114981 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:38.115019 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:38.115808 (+ 789us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:38.116426 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:38.116432 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:38.116851 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:38.117461 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:38.117480 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:38.118546 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:38.118550 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:38.119968 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:38.119980 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:38.120609 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:38.120614 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:38.120805 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:38.121381 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:31:38.121402 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:38.123106 (+ 1704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:38.124955 (+ 1849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:38.124960 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:38.124973 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:38.125336 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:38.125575 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:38.125576 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:38.125578 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:38.125693 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:38.125989 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:38.125992 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:38.126210 (+ 218us) client_negotiation.cc:769] Sending connection context
0808 22:31:38.126346 (+ 136us) client_negotiation.cc:240] Negotiation successful
0808 22:31:38.126568 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":130,"threads_started":1}
I20260808 22:31:38.128470 1983 heartbeater.cc:347] Connected to a master server at 127.31.87.190:39551
I20260808 22:31:38.128602 1983 heartbeater.cc:464] Registering TS with master...
I20260808 22:31:38.129047 1983 heartbeater.cc:511] Master 127.31.87.190:39551 requested a full tablet report, sending...
I20260808 22:31:38.130095 1381 ts_manager.cc:194] Registered new tserver with Master: dba5657cda984625a81e9fa4dda06b8b (127.31.87.130:44213)
I20260808 22:31:38.130859 1381 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.130@KRBTEST.COM'} at 127.31.87.130:56627
I20260808 22:31:38.130946 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 1756
I20260808 22:31:38.131042 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-1/wal/instance
I20260808 22:31:38.131335 32094 external_mini_cluster.cc:1428] Setting key 170597b16ebe906d1e91991e4ed7bcf2
I20260808 22:31:38.137209 1472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:38.130410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:60569 (local address 127.31.87.190:39551)
0808 22:31:38.130527 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:38.130531 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:38.130715 (+ 184us) server_negotiation.cc:407] Connection header received
0808 22:31:38.130810 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:38.130813 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:38.130859 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:38.130938 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:38.131389 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:38.131918 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:38.132855 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:38.133183 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:38.133257 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:38.134425 (+ 1168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:38.134428 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:38.134429 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:38.134451 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:38.135689 (+ 1238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:38.136090 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:38.136092 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:38.136093 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:38.136156 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:38.136422 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:38.136426 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:38.136428 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:38.136618 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:38.136702 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:38.136947 (+ 245us) server_negotiation.cc:299] Negotiation successful
0808 22:31:38.137045 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 22:31:38.137218 1992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:38.130358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:39551 (local address 127.31.87.130:60569)
0808 22:31:38.130637 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:38.130650 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:38.130740 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:38.130967 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:38.130969 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:38.131175 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:38.131279 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:38.131284 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:38.132028 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:38.132032 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:38.132719 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:38.132731 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:38.133397 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:38.133400 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:38.133482 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:38.133912 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:31:38.133925 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:38.134300 (+ 375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:38.135790 (+ 1490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:38.135794 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:38.135796 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:38.136015 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:38.136241 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:38.136245 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:38.136247 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:38.136341 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:38.136708 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:38.136711 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:38.136799 (+ 88us) client_negotiation.cc:769] Sending connection context
0808 22:31:38.136884 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 22:31:38.137008 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"mutex_wait_us":49,"thread_start_us":109,"threads_started":1}
WARNING: no policy specified for kudu/127.31.87.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab.
Entry for principal kudu/127.31.87.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab.
I20260808 22:31:38.157780 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.131:0
--local_ip_for_outbound_sockets=127.31.87.131
--webserver_interface=127.31.87.131
--webserver_port=0
--tserver_master_addrs=127.31.87.190:39551
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab
--principal=kudu/127.31.87.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:39701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:31:38.278120 1995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:38.278368 1995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:38.278438 1995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:38.282428 1995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:38.282485 1995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:38.282548 1995 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 22:31:38.282613 1995 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.131
I20260808 22:31:38.287671 1995 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:39701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab
--principal=kudu/127.31.87.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.87.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.87.190:39551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1995
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.131
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:38.288656 1995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:38.289691 1995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:38.298974 2011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:38.299010 2005 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:38.298974 2003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:38.299273 1995 server_base.cc:1034] running on GCE node
I20260808 22:31:38.299695 1995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:38.300292 1995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:38.301443 1995 hybrid_clock.cc:648] HybridClock initialized: now 1786228298301426 us; error 30 us; skew 500 ppm
Aug 08 22:31:38 dist-test-slave-266d krb5kdc[1318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228298, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:31:38.304124 1995 init.cc:377] Logged in from keytab as kudu/127.31.87.131@KRBTEST.COM (short username kudu)
I20260808 22:31:38.305004 1995 webserver.cc:460] Webserver started at http://127.31.87.131:43109/ using document root <none> and password file <none>
I20260808 22:31:38.305552 1995 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:38.305611 1995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:38.305806 1995 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:38.307691 1995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-2/data/instance:
uuid: "762affbd72ce479ca38322dc19ef2a80"
format_stamp: "Formatted at 2026-08-08 22:31:38 on dist-test-slave-266d"
server_key: "3b6a22eb0a52da933e5df6cb99748460"
server_key_iv: "dcf0050ba3bf119cc5429ba260ad1396"
server_key_version: "encryptionkey@0"
I20260808 22:31:38.308221 1995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-2/wal/instance:
uuid: "762affbd72ce479ca38322dc19ef2a80"
format_stamp: "Formatted at 2026-08-08 22:31:38 on dist-test-slave-266d"
server_key: "3b6a22eb0a52da933e5df6cb99748460"
server_key_iv: "dcf0050ba3bf119cc5429ba260ad1396"
server_key_version: "encryptionkey@0"
I20260808 22:31:38.311781 1995 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260808 22:31:38.314167 2021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:38.315280 1995 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:31:38.315375 1995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-2/wal
uuid: "762affbd72ce479ca38322dc19ef2a80"
format_stamp: "Formatted at 2026-08-08 22:31:38 on dist-test-slave-266d"
server_key: "3b6a22eb0a52da933e5df6cb99748460"
server_key_iv: "dcf0050ba3bf119cc5429ba260ad1396"
server_key_version: "encryptionkey@0"
I20260808 22:31:38.315466 1995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:38.337939 1995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:38.340907 1995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:38.341070 1995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:38.342486 1995 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:31:38.342535 1995 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:38.342576 1995 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:31:38.342604 1995 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 22:31:38 dist-test-slave-266d krb5kdc[1318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228298, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.131@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:31:38.355473 1995 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.131:38801
I20260808 22:31:38.355489 2213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.131:38801 every 8 connection(s)
I20260808 22:31:38.356541 1995 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
I20260808 22:31:38.356871 1472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:38.343276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:40385 (local address 127.31.87.190:39551)
0808 22:31:38.343397 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:38.343402 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:38.344411 (+ 1009us) server_negotiation.cc:407] Connection header received
0808 22:31:38.345498 (+ 1087us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:38.345502 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:38.345564 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:38.345648 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:38.347327 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:38.348132 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:38.349873 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:38.350221 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:38.350370 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:38.353010 (+ 2640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:38.353013 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:38.353015 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:38.353043 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:38.354697 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:38.355245 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:38.355249 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:38.355251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:38.355361 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:38.355765 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:38.355769 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:38.355771 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:38.356002 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:38.356106 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:38.356586 (+ 480us) server_negotiation.cc:299] Negotiation successful
0808 22:31:38.356714 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 22:31:38.357662 2038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:38.343795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:39551 (local address 127.31.87.131:40385)
0808 22:31:38.344257 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:38.344298 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:38.345244 (+ 946us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:38.346049 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:38.346058 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:38.346479 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:38.347124 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:38.347143 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:38.348276 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:38.348279 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:38.349721 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:38.349736 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:38.350427 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:38.350431 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:38.350632 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:38.351225 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:31:38.351242 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:38.352837 (+ 1595us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:38.354825 (+ 1988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:38.354831 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:38.354845 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:38.355147 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:38.355462 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:38.355465 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:38.355467 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:38.355671 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:38.356093 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:38.356098 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:38.356348 (+ 250us) client_negotiation.cc:769] Sending connection context
0808 22:31:38.356502 (+ 154us) client_negotiation.cc:240] Negotiation successful
0808 22:31:38.356752 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":130,"threads_started":1}
I20260808 22:31:38.358814 2214 heartbeater.cc:347] Connected to a master server at 127.31.87.190:39551
I20260808 22:31:38.358948 2214 heartbeater.cc:464] Registering TS with master...
I20260808 22:31:38.359416 2214 heartbeater.cc:511] Master 127.31.87.190:39551 requested a full tablet report, sending...
I20260808 22:31:38.360453 1378 ts_manager.cc:194] Registered new tserver with Master: 762affbd72ce479ca38322dc19ef2a80 (127.31.87.131:38801)
I20260808 22:31:38.361001 1378 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.131@KRBTEST.COM'} at 127.31.87.131:40385
I20260808 22:31:38.363479 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 1995
I20260808 22:31:38.363592 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-2/wal/instance
I20260808 22:31:38.363842 32094 external_mini_cluster.cc:1428] Setting key 114008c12078f0b91477dce1b35eae4a
I20260808 22:31:38.365808 32094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:31:38.369275 1472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:38.361419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:45257 (local address 127.31.87.190:39551)
0808 22:31:38.361538 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:38.361542 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:38.362843 (+ 1301us) server_negotiation.cc:407] Connection header received
0808 22:31:38.362916 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:38.362918 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:38.362956 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:38.363018 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:38.363414 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:38.364012 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:38.364858 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:38.365077 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:38.365190 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:38.366314 (+ 1124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:38.366317 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:38.366319 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:38.366340 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:38.367636 (+ 1296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:38.368120 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:38.368124 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:38.368125 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:38.368215 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:38.368473 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:38.368477 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:38.368479 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:38.368644 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:38.368729 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:38.368978 (+ 249us) server_negotiation.cc:299] Negotiation successful
0808 22:31:38.369114 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 22:31:38.369289 2224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:38.361339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:39551 (local address 127.31.87.131:45257)
0808 22:31:38.362752 (+ 1413us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:38.362775 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:38.362858 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:38.363039 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:38.363041 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:38.363206 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:38.363303 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:38.363308 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:38.364127 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:38.364131 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:38.364748 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:38.364758 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:38.365193 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:38.365197 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:38.365285 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:38.365781 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:31:38.365796 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:38.366193 (+ 397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:38.367740 (+ 1547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:38.367744 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:38.367746 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:38.368001 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:38.368317 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:38.368319 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:38.368321 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:38.368389 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:38.368732 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:38.368735 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:38.368824 (+ 89us) client_negotiation.cc:769] Sending connection context
0808 22:31:38.368909 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 22:31:38.369053 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1342,"thread_start_us":90,"threads_started":1}
Aug 08 22:31:38 dist-test-slave-266d krb5kdc[1318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228298, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 22:31:38 dist-test-slave-266d krb5kdc[1318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228298, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:31:38.392390 1472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:38.385455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58020 (local address 127.31.87.190:39551)
0808 22:31:38.385573 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:38.385576 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:38.385588 (+ 12us) server_negotiation.cc:407] Connection header received
0808 22:31:38.385628 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:38.385630 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:38.385667 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:38.385734 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:38.386339 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:38.386844 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:38.387564 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:38.387759 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:38.387822 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:38.389959 (+ 2137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:38.389962 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:38.389964 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:38.389986 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:38.391153 (+ 1167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:38.391524 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:38.391526 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:38.391528 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:38.391596 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:38.391789 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:38.391791 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:38.391792 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:38.391941 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:38.392032 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:38.392182 (+ 150us) server_negotiation.cc:299] Negotiation successful
0808 22:31:38.392268 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 22:31:38.395269 1378 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58020:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:31:38.398188 1378 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:31:38.412436 2243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:38.406406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.130:44213 (local address 127.0.0.1:33162)
0808 22:31:38.407251 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:38.407312 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:38.407448 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:38.408216 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:38.408221 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:38.408240 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:31:38.408371 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:38.408377 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:38.409715 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:38.409718 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:38.410750 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:38.410761 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:38.411849 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:38.411853 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:38.411966 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:38.412010 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 22:31:38.412131 (+ 121us) client_negotiation.cc:240] Negotiation successful
0808 22:31:38.412260 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":675,"mutex_wait_us":32,"spinlock_wait_cycles":3584,"thread_start_us":64,"threads_started":1}
I20260808 22:31:38.412616 2244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:38.406545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33162 (local address 127.31.87.130:44213)
0808 22:31:38.407744 (+ 1199us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:38.407756 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:38.407779 (+ 23us) server_negotiation.cc:407] Connection header received
0808 22:31:38.407853 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:38.407859 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:38.408061 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:38.408228 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:31:38.408509 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:38.409601 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:38.410880 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:38.411723 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:38.411840 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:38.411989 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:38.412184 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 22:31:38.412460 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1087,"thread_start_us":136,"threads_started":1}
I20260808 22:31:38.413115 2252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:38.407554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48938 (local address 127.31.87.129:40357)
0808 22:31:38.407857 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:38.407862 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:38.408558 (+ 696us) server_negotiation.cc:407] Connection header received
0808 22:31:38.408707 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:38.408713 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:38.408901 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:38.409034 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:31:38.409314 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:38.410498 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:38.411458 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:38.412096 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:38.412185 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:38.412326 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:38.412861 (+ 535us) server_negotiation.cc:299] Negotiation successful
0808 22:31:38.412962 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":125,"threads_started":1}
I20260808 22:31:38.412753 2247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:38.406755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:40357 (local address 127.0.0.1:48938)
0808 22:31:38.408466 (+ 1711us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:38.408480 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:38.408575 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:38.409035 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:38.409038 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:38.409057 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:31:38.409208 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:38.409214 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:38.410609 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:38.410612 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:38.411346 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:38.411354 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:38.412387 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:38.412391 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:38.412483 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:38.412505 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 22:31:38.412575 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 22:31:38.412641 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1653,"thread_start_us":71,"threads_started":1}
I20260808 22:31:38.413406 2248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:38.406529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.131:38801 (local address 127.0.0.1:48584)
0808 22:31:38.407265 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:38.407313 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:38.407449 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:38.408389 (+ 940us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:38.408391 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:38.408407 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:31:38.408478 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:38.408483 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:38.409592 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:38.409595 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:38.411726 (+ 2131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:38.411735 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:38.412967 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:38.412971 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:38.413058 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:38.413079 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 22:31:38.413193 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 22:31:38.413269 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":675,"mutex_wait_us":56,"spinlock_wait_cycles":7296,"thread_start_us":139,"threads_started":1}
I20260808 22:31:38.413727 2251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:38.407180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48584 (local address 127.31.87.131:38801)
0808 22:31:38.407992 (+ 812us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:38.408002 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:38.408022 (+ 20us) server_negotiation.cc:407] Connection header received
0808 22:31:38.408088 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:38.408093 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:38.408258 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:38.408401 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:31:38.408634 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:38.409499 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:38.412053 (+ 2554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:38.412844 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:38.412951 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:38.413135 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:38.413495 (+ 360us) server_negotiation.cc:299] Negotiation successful
0808 22:31:38.413594 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":722,"thread_start_us":113,"threads_started":1}
I20260808 22:31:38.416030 1863 tablet_service.cc:1467] Processing CreateTablet for tablet c88c7b39e9134688afbf777e41aeaade (DEFAULT_TABLE table=test-table [id=08a3e16adb0f408192b216a43a1e1126]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:31:38.416527 2107 tablet_service.cc:1467] Processing CreateTablet for tablet c88c7b39e9134688afbf777e41aeaade (DEFAULT_TABLE table=test-table [id=08a3e16adb0f408192b216a43a1e1126]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:31:38.416525 1631 tablet_service.cc:1467] Processing CreateTablet for tablet c88c7b39e9134688afbf777e41aeaade (DEFAULT_TABLE table=test-table [id=08a3e16adb0f408192b216a43a1e1126]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:31:38.417181 1863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c88c7b39e9134688afbf777e41aeaade. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:38.417615 1631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c88c7b39e9134688afbf777e41aeaade. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:38.417629 2107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c88c7b39e9134688afbf777e41aeaade. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:38.424562 2253 tablet_bootstrap.cc:492] T c88c7b39e9134688afbf777e41aeaade P dba5657cda984625a81e9fa4dda06b8b: Bootstrap starting.
I20260808 22:31:38.425804 2254 tablet_bootstrap.cc:492] T c88c7b39e9134688afbf777e41aeaade P 9f36d03373d54b269ac978d372329799: Bootstrap starting.
I20260808 22:31:38.426192 2256 tablet_bootstrap.cc:492] T c88c7b39e9134688afbf777e41aeaade P 762affbd72ce479ca38322dc19ef2a80: Bootstrap starting.
I20260808 22:31:38.426963 2253 tablet_bootstrap.cc:654] T c88c7b39e9134688afbf777e41aeaade P dba5657cda984625a81e9fa4dda06b8b: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:38.427845 2253 log.cc:824] T c88c7b39e9134688afbf777e41aeaade P dba5657cda984625a81e9fa4dda06b8b: Log is configured to *not* fsync() on all Append() calls
I20260808 22:31:38.428109 2256 tablet_bootstrap.cc:654] T c88c7b39e9134688afbf777e41aeaade P 762affbd72ce479ca38322dc19ef2a80: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:38.428144 2254 tablet_bootstrap.cc:654] T c88c7b39e9134688afbf777e41aeaade P 9f36d03373d54b269ac978d372329799: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:38.428996 2254 log.cc:824] T c88c7b39e9134688afbf777e41aeaade P 9f36d03373d54b269ac978d372329799: Log is configured to *not* fsync() on all Append() calls
I20260808 22:31:38.428996 2256 log.cc:824] T c88c7b39e9134688afbf777e41aeaade P 762affbd72ce479ca38322dc19ef2a80: Log is configured to *not* fsync() on all Append() calls
I20260808 22:31:38.430217 2253 tablet_bootstrap.cc:492] T c88c7b39e9134688afbf777e41aeaade P dba5657cda984625a81e9fa4dda06b8b: No bootstrap required, opened a new log
I20260808 22:31:38.430413 2253 ts_tablet_manager.cc:1397] T c88c7b39e9134688afbf777e41aeaade P dba5657cda984625a81e9fa4dda06b8b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 22:31:38.431052 2256 tablet_bootstrap.cc:492] T c88c7b39e9134688afbf777e41aeaade P 762affbd72ce479ca38322dc19ef2a80: No bootstrap required, opened a new log
I20260808 22:31:38.431044 2254 tablet_bootstrap.cc:492] T c88c7b39e9134688afbf777e41aeaade P 9f36d03373d54b269ac978d372329799: No bootstrap required, opened a new log
I20260808 22:31:38.431237 2254 ts_tablet_manager.cc:1397] T c88c7b39e9134688afbf777e41aeaade P 9f36d03373d54b269ac978d372329799: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 22:31:38.431237 2256 ts_tablet_manager.cc:1397] T c88c7b39e9134688afbf777e41aeaade P 762affbd72ce479ca38322dc19ef2a80: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 22:31:38.433471 2253 raft_consensus.cc:348] T c88c7b39e9134688afbf777e41aeaade P dba5657cda984625a81e9fa4dda06b8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dba5657cda984625a81e9fa4dda06b8b" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 44213 } } peers { permanent_uuid: "762affbd72ce479ca38322dc19ef2a80" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 38801 } } peers { permanent_uuid: "9f36d03373d54b269ac978d372329799" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40357 } }
I20260808 22:31:38.433712 2253 raft_consensus.cc:374] T c88c7b39e9134688afbf777e41aeaade P dba5657cda984625a81e9fa4dda06b8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:38.433799 2253 raft_consensus.cc:729] T c88c7b39e9134688afbf777e41aeaade P dba5657cda984625a81e9fa4dda06b8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dba5657cda984625a81e9fa4dda06b8b, State: Initialized, Role: FOLLOWER
I20260808 22:31:38.434259 2253 consensus_queue.cc:260] T c88c7b39e9134688afbf777e41aeaade P dba5657cda984625a81e9fa4dda06b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dba5657cda984625a81e9fa4dda06b8b" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 44213 } } peers { permanent_uuid: "762affbd72ce479ca38322dc19ef2a80" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 38801 } } peers { permanent_uuid: "9f36d03373d54b269ac978d372329799" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40357 } }
I20260808 22:31:38.435361 2253 ts_tablet_manager.cc:1428] T c88c7b39e9134688afbf777e41aeaade P dba5657cda984625a81e9fa4dda06b8b: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 22:31:38.435295 2256 raft_consensus.cc:348] T c88c7b39e9134688afbf777e41aeaade P 762affbd72ce479ca38322dc19ef2a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dba5657cda984625a81e9fa4dda06b8b" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 44213 } } peers { permanent_uuid: "762affbd72ce479ca38322dc19ef2a80" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 38801 } } peers { permanent_uuid: "9f36d03373d54b269ac978d372329799" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40357 } }
I20260808 22:31:38.435286 2254 raft_consensus.cc:348] T c88c7b39e9134688afbf777e41aeaade P 9f36d03373d54b269ac978d372329799 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dba5657cda984625a81e9fa4dda06b8b" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 44213 } } peers { permanent_uuid: "762affbd72ce479ca38322dc19ef2a80" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 38801 } } peers { permanent_uuid: "9f36d03373d54b269ac978d372329799" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40357 } }
I20260808 22:31:38.435552 2254 raft_consensus.cc:374] T c88c7b39e9134688afbf777e41aeaade P 9f36d03373d54b269ac978d372329799 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:38.435577 2256 raft_consensus.cc:374] T c88c7b39e9134688afbf777e41aeaade P 762affbd72ce479ca38322dc19ef2a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:38.435634 2254 raft_consensus.cc:729] T c88c7b39e9134688afbf777e41aeaade P 9f36d03373d54b269ac978d372329799 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f36d03373d54b269ac978d372329799, State: Initialized, Role: FOLLOWER
I20260808 22:31:38.435781 1983 heartbeater.cc:503] Master 127.31.87.190:39551 was elected leader, sending a full tablet report...
I20260808 22:31:38.436142 2254 consensus_queue.cc:260] T c88c7b39e9134688afbf777e41aeaade P 9f36d03373d54b269ac978d372329799 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dba5657cda984625a81e9fa4dda06b8b" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 44213 } } peers { permanent_uuid: "762affbd72ce479ca38322dc19ef2a80" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 38801 } } peers { permanent_uuid: "9f36d03373d54b269ac978d372329799" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40357 } }
I20260808 22:31:38.436940 1743 heartbeater.cc:503] Master 127.31.87.190:39551 was elected leader, sending a full tablet report...
I20260808 22:31:38.436977 2256 raft_consensus.cc:729] T c88c7b39e9134688afbf777e41aeaade P 762affbd72ce479ca38322dc19ef2a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 762affbd72ce479ca38322dc19ef2a80, State: Initialized, Role: FOLLOWER
I20260808 22:31:38.437340 2254 ts_tablet_manager.cc:1428] T c88c7b39e9134688afbf777e41aeaade P 9f36d03373d54b269ac978d372329799: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 22:31:38.437512 2256 consensus_queue.cc:260] T c88c7b39e9134688afbf777e41aeaade P 762affbd72ce479ca38322dc19ef2a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dba5657cda984625a81e9fa4dda06b8b" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 44213 } } peers { permanent_uuid: "762affbd72ce479ca38322dc19ef2a80" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 38801 } } peers { permanent_uuid: "9f36d03373d54b269ac978d372329799" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40357 } }
I20260808 22:31:38.438160 2214 heartbeater.cc:503] Master 127.31.87.190:39551 was elected leader, sending a full tablet report...
I20260808 22:31:38.438501 2256 ts_tablet_manager.cc:1428] T c88c7b39e9134688afbf777e41aeaade P 762affbd72ce479ca38322dc19ef2a80: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260808 22:31:38.460691 2266 raft_consensus.cc:482] T c88c7b39e9134688afbf777e41aeaade P 762affbd72ce479ca38322dc19ef2a80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:31:38.460954 2266 raft_consensus.cc:504] T c88c7b39e9134688afbf777e41aeaade P 762affbd72ce479ca38322dc19ef2a80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dba5657cda984625a81e9fa4dda06b8b" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 44213 } } peers { permanent_uuid: "762affbd72ce479ca38322dc19ef2a80" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 38801 } } peers { permanent_uuid: "9f36d03373d54b269ac978d372329799" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40357 } }
I20260808 22:31:38.462218 2266 leader_election.cc:288] T c88c7b39e9134688afbf777e41aeaade P 762affbd72ce479ca38322dc19ef2a80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dba5657cda984625a81e9fa4dda06b8b (127.31.87.130:44213), 9f36d03373d54b269ac978d372329799 (127.31.87.129:40357)
I20260808 22:31:38.466648 2038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:38.462473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.130:44213 (local address 127.31.87.131:53277)
0808 22:31:38.462694 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:38.462709 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:38.462810 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:38.463053 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:38.463056 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:38.463070 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:31:38.463188 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:38.463194 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:38.464199 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:38.464203 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:38.465501 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:38.465515 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:38.466274 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:38.466277 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:38.466368 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:38.466389 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 22:31:38.466446 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 22:31:38.466505 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":21}
I20260808 22:31:38.466682 2244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:38.462665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:53277 (local address 127.31.87.130:44213)
0808 22:31:38.462802 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:38.462807 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:38.462824 (+ 17us) server_negotiation.cc:407] Connection header received
0808 22:31:38.462908 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:38.462910 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:38.462953 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:38.463026 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:31:38.463294 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:38.464068 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:38.465652 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:38.466125 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:38.466283 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:38.466338 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:38.466475 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 22:31:38.466546 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 22:31:38.467337 2252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:38.462626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:45385 (local address 127.31.87.129:40357)
0808 22:31:38.462742 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:38.462746 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:38.463051 (+ 305us) server_negotiation.cc:407] Connection header received
0808 22:31:38.463141 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:38.463144 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:38.463186 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:38.463247 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:31:38.463496 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:38.464224 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:38.466055 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:38.466634 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:38.466763 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:38.466810 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:38.467126 (+ 316us) server_negotiation.cc:299] Negotiation successful
0808 22:31:38.467212 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 22:31:38.467379 2267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:38.462531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:40357 (local address 127.31.87.131:45385)
0808 22:31:38.462953 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:38.462964 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:38.463057 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:38.463277 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:38.463280 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:38.463300 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:31:38.463397 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:38.463401 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:38.464319 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:38.464322 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:38.465928 (+ 1606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:38.465937 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:38.466892 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:38.466896 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:38.467003 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:38.467027 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 22:31:38.467096 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 22:31:38.467165 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"mutex_wait_us":1,"thread_start_us":164,"threads_started":1}
I20260808 22:31:38.467557 1900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c88c7b39e9134688afbf777e41aeaade" candidate_uuid: "762affbd72ce479ca38322dc19ef2a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dba5657cda984625a81e9fa4dda06b8b" is_pre_election: true
I20260808 22:31:38.467914 1900 raft_consensus.cc:2454] T c88c7b39e9134688afbf777e41aeaade P dba5657cda984625a81e9fa4dda06b8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 762affbd72ce479ca38322dc19ef2a80 in term 0.
I20260808 22:31:38.468086 1660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c88c7b39e9134688afbf777e41aeaade" candidate_uuid: "762affbd72ce479ca38322dc19ef2a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f36d03373d54b269ac978d372329799" is_pre_election: true
I20260808 22:31:38.468389 1660 raft_consensus.cc:2454] T c88c7b39e9134688afbf777e41aeaade P 9f36d03373d54b269ac978d372329799 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 762affbd72ce479ca38322dc19ef2a80 in term 0.
I20260808 22:31:38.468428 2026 leader_election.cc:302] T c88c7b39e9134688afbf777e41aeaade P 762affbd72ce479ca38322dc19ef2a80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 762affbd72ce479ca38322dc19ef2a80, dba5657cda984625a81e9fa4dda06b8b; no voters:
I20260808 22:31:38.468704 2266 raft_consensus.cc:2781] T c88c7b39e9134688afbf777e41aeaade P 762affbd72ce479ca38322dc19ef2a80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:31:38.468781 2266 raft_consensus.cc:482] T c88c7b39e9134688afbf777e41aeaade P 762affbd72ce479ca38322dc19ef2a80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:31:38.468827 2266 raft_consensus.cc:3037] T c88c7b39e9134688afbf777e41aeaade P 762affbd72ce479ca38322dc19ef2a80 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:38.469916 2266 raft_consensus.cc:504] T c88c7b39e9134688afbf777e41aeaade P 762affbd72ce479ca38322dc19ef2a80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dba5657cda984625a81e9fa4dda06b8b" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 44213 } } peers { permanent_uuid: "762affbd72ce479ca38322dc19ef2a80" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 38801 } } peers { permanent_uuid: "9f36d03373d54b269ac978d372329799" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40357 } }
I20260808 22:31:38.470283 2266 leader_election.cc:288] T c88c7b39e9134688afbf777e41aeaade P 762affbd72ce479ca38322dc19ef2a80 [CANDIDATE]: Term 1 election: Requested vote from peers dba5657cda984625a81e9fa4dda06b8b (127.31.87.130:44213), 9f36d03373d54b269ac978d372329799 (127.31.87.129:40357)
I20260808 22:31:38.470633 1900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c88c7b39e9134688afbf777e41aeaade" candidate_uuid: "762affbd72ce479ca38322dc19ef2a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dba5657cda984625a81e9fa4dda06b8b"
I20260808 22:31:38.470654 1660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c88c7b39e9134688afbf777e41aeaade" candidate_uuid: "762affbd72ce479ca38322dc19ef2a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f36d03373d54b269ac978d372329799"
I20260808 22:31:38.470767 1660 raft_consensus.cc:3037] T c88c7b39e9134688afbf777e41aeaade P 9f36d03373d54b269ac978d372329799 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:38.470750 1900 raft_consensus.cc:3037] T c88c7b39e9134688afbf777e41aeaade P dba5657cda984625a81e9fa4dda06b8b [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:38.471856 1900 raft_consensus.cc:2454] T c88c7b39e9134688afbf777e41aeaade P dba5657cda984625a81e9fa4dda06b8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 762affbd72ce479ca38322dc19ef2a80 in term 1.
I20260808 22:31:38.471868 1660 raft_consensus.cc:2454] T c88c7b39e9134688afbf777e41aeaade P 9f36d03373d54b269ac978d372329799 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 762affbd72ce479ca38322dc19ef2a80 in term 1.
I20260808 22:31:38.472179 2022 leader_election.cc:302] T c88c7b39e9134688afbf777e41aeaade P 762affbd72ce479ca38322dc19ef2a80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 762affbd72ce479ca38322dc19ef2a80, 9f36d03373d54b269ac978d372329799, dba5657cda984625a81e9fa4dda06b8b; no voters:
I20260808 22:31:38.472375 2266 raft_consensus.cc:2781] T c88c7b39e9134688afbf777e41aeaade P 762affbd72ce479ca38322dc19ef2a80 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:31:38.472594 2266 raft_consensus.cc:686] T c88c7b39e9134688afbf777e41aeaade P 762affbd72ce479ca38322dc19ef2a80 [term 1 LEADER]: Becoming Leader. State: Replica: 762affbd72ce479ca38322dc19ef2a80, State: Running, Role: LEADER
I20260808 22:31:38.472929 2266 consensus_queue.cc:237] T c88c7b39e9134688afbf777e41aeaade P 762affbd72ce479ca38322dc19ef2a80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dba5657cda984625a81e9fa4dda06b8b" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 44213 } } peers { permanent_uuid: "762affbd72ce479ca38322dc19ef2a80" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 38801 } } peers { permanent_uuid: "9f36d03373d54b269ac978d372329799" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40357 } }
I20260808 22:31:38.476564 1381 catalog_manager.cc:5495] T c88c7b39e9134688afbf777e41aeaade P 762affbd72ce479ca38322dc19ef2a80 reported cstate change: term changed from 0 to 1, leader changed from <none> to 762affbd72ce479ca38322dc19ef2a80 (127.31.87.131). New cstate: current_term: 1 leader_uuid: "762affbd72ce479ca38322dc19ef2a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dba5657cda984625a81e9fa4dda06b8b" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 44213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "762affbd72ce479ca38322dc19ef2a80" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 38801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9f36d03373d54b269ac978d372329799" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40357 } health_report { overall_health: UNKNOWN } } }
I20260808 22:31:38.499095 2251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:38.495324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48592 (local address 127.31.87.131:38801)
0808 22:31:38.495448 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:38.495453 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:38.495581 (+ 128us) server_negotiation.cc:407] Connection header received
0808 22:31:38.495810 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:38.495813 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:38.495871 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:38.495955 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:31:38.496209 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:38.496722 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:38.497658 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:38.497878 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:38.497966 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:38.498283 (+ 317us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:31:38.498701 (+ 418us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:31:38.498811 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:38.498896 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 22:31:38.498958 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 22:31:38.505200 1900 raft_consensus.cc:1261] T c88c7b39e9134688afbf777e41aeaade P dba5657cda984625a81e9fa4dda06b8b [term 1 FOLLOWER]: Refusing update from remote peer 762affbd72ce479ca38322dc19ef2a80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:31:38.505934 2269 consensus_queue.cc:1035] T c88c7b39e9134688afbf777e41aeaade P 762affbd72ce479ca38322dc19ef2a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dba5657cda984625a81e9fa4dda06b8b" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 44213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:31:38.507021 1660 raft_consensus.cc:1261] T c88c7b39e9134688afbf777e41aeaade P 9f36d03373d54b269ac978d372329799 [term 1 FOLLOWER]: Refusing update from remote peer 762affbd72ce479ca38322dc19ef2a80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:31:38.507581 2269 consensus_queue.cc:1035] T c88c7b39e9134688afbf777e41aeaade P 762affbd72ce479ca38322dc19ef2a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f36d03373d54b269ac978d372329799" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:31:38.513532 2275 mvcc.cc:205] Tried to move back new op lower bound from 7316391110668869632 to 7316391110548246528. Current Snapshot: MvccSnapshot[applied={T|T < 7316391110668869632}]
I20260808 22:31:38.526516 1381 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:31:38.526818 1381 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:31:38.529803 1381 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 989bdf9e19a44aa3987414387310f4b9: Sending DeleteTablet for 3 replicas of tablet c88c7b39e9134688afbf777e41aeaade
I20260808 22:31:38.530627 2107 tablet_service.cc:1514] Processing DeleteTablet for tablet c88c7b39e9134688afbf777e41aeaade with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:31:38 UTC) from {username='kudu', principal='kudu/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:48584
I20260808 22:31:38.530787 1863 tablet_service.cc:1514] Processing DeleteTablet for tablet c88c7b39e9134688afbf777e41aeaade with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:31:38 UTC) from {username='kudu', principal='kudu/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:33162
I20260808 22:31:38.531311 2288 tablet_replica.cc:330] T c88c7b39e9134688afbf777e41aeaade P dba5657cda984625a81e9fa4dda06b8b: stopping tablet replica
I20260808 22:31:38.531306 1631 tablet_service.cc:1514] Processing DeleteTablet for tablet c88c7b39e9134688afbf777e41aeaade with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:31:38 UTC) from {username='kudu', principal='kudu/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:48938
I20260808 22:31:38.531405 2285 tablet_replica.cc:330] T c88c7b39e9134688afbf777e41aeaade P 762affbd72ce479ca38322dc19ef2a80: stopping tablet replica
I20260808 22:31:38.531718 2288 raft_consensus.cc:2229] T c88c7b39e9134688afbf777e41aeaade P dba5657cda984625a81e9fa4dda06b8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:31:38.531898 2285 raft_consensus.cc:2229] T c88c7b39e9134688afbf777e41aeaade P 762affbd72ce479ca38322dc19ef2a80 [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:31:38.532147 2288 raft_consensus.cc:2258] T c88c7b39e9134688afbf777e41aeaade P dba5657cda984625a81e9fa4dda06b8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:31:38.532368 2285 raft_consensus.cc:2258] T c88c7b39e9134688afbf777e41aeaade P 762affbd72ce479ca38322dc19ef2a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:31:38.532610 1381 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58020:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:31:38.533026 1381 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:31:38.534060 2290 tablet_replica.cc:330] T c88c7b39e9134688afbf777e41aeaade P 9f36d03373d54b269ac978d372329799: stopping tablet replica
I20260808 22:31:38.534096 2288 ts_tablet_manager.cc:1905] T c88c7b39e9134688afbf777e41aeaade P dba5657cda984625a81e9fa4dda06b8b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:31:38.534401 2290 raft_consensus.cc:2229] T c88c7b39e9134688afbf777e41aeaade P 9f36d03373d54b269ac978d372329799 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:31:38.534734 2290 raft_consensus.cc:2258] T c88c7b39e9134688afbf777e41aeaade P 9f36d03373d54b269ac978d372329799 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:31:38.535878 2285 ts_tablet_manager.cc:1905] T c88c7b39e9134688afbf777e41aeaade P 762affbd72ce479ca38322dc19ef2a80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:31:38.537330 2288 ts_tablet_manager.cc:1918] T c88c7b39e9134688afbf777e41aeaade P dba5657cda984625a81e9fa4dda06b8b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:31:38.537500 2288 log.cc:1196] T c88c7b39e9134688afbf777e41aeaade P dba5657cda984625a81e9fa4dda06b8b: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-1/wal/wals/c88c7b39e9134688afbf777e41aeaade
I20260808 22:31:38.537936 2288 ts_tablet_manager.cc:1939] T c88c7b39e9134688afbf777e41aeaade P dba5657cda984625a81e9fa4dda06b8b: Deleting consensus metadata
I20260808 22:31:38.538553 2285 ts_tablet_manager.cc:1918] T c88c7b39e9134688afbf777e41aeaade P 762affbd72ce479ca38322dc19ef2a80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:31:38.538694 2285 log.cc:1196] T c88c7b39e9134688afbf777e41aeaade P 762affbd72ce479ca38322dc19ef2a80: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-2/wal/wals/c88c7b39e9134688afbf777e41aeaade
I20260808 22:31:38.539006 2285 ts_tablet_manager.cc:1939] T c88c7b39e9134688afbf777e41aeaade P 762affbd72ce479ca38322dc19ef2a80: Deleting consensus metadata
I20260808 22:31:38.539397 1360 catalog_manager.cc:4841] TS dba5657cda984625a81e9fa4dda06b8b (127.31.87.130:44213): tablet c88c7b39e9134688afbf777e41aeaade (table test-table [id=08a3e16adb0f408192b216a43a1e1126]) successfully deleted
I20260808 22:31:38.540102 1362 catalog_manager.cc:4841] TS 762affbd72ce479ca38322dc19ef2a80 (127.31.87.131:38801): tablet c88c7b39e9134688afbf777e41aeaade (table test-table [id=08a3e16adb0f408192b216a43a1e1126]) successfully deleted
I20260808 22:31:38.540653 1631 tablet_service.cc:1467] Processing CreateTablet for tablet d1aa1be2c714429d9deafd5f00abeeed (DEFAULT_TABLE table=test-table [id=6145aa535915492ba6da1a9576344bf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:31:38.540915 1631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1aa1be2c714429d9deafd5f00abeeed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:38.540910 1863 tablet_service.cc:1467] Processing CreateTablet for tablet d1aa1be2c714429d9deafd5f00abeeed (DEFAULT_TABLE table=test-table [id=6145aa535915492ba6da1a9576344bf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:31:38.541194 1863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1aa1be2c714429d9deafd5f00abeeed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:38.541484 2290 ts_tablet_manager.cc:1905] T c88c7b39e9134688afbf777e41aeaade P 9f36d03373d54b269ac978d372329799: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:31:38.543339 2254 tablet_bootstrap.cc:492] T d1aa1be2c714429d9deafd5f00abeeed P 9f36d03373d54b269ac978d372329799: Bootstrap starting.
I20260808 22:31:38.543403 2253 tablet_bootstrap.cc:492] T d1aa1be2c714429d9deafd5f00abeeed P dba5657cda984625a81e9fa4dda06b8b: Bootstrap starting.
I20260808 22:31:38.543970 2290 ts_tablet_manager.cc:1918] T c88c7b39e9134688afbf777e41aeaade P 9f36d03373d54b269ac978d372329799: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:31:38.544041 2290 log.cc:1196] T c88c7b39e9134688afbf777e41aeaade P 9f36d03373d54b269ac978d372329799: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-0/wal/wals/c88c7b39e9134688afbf777e41aeaade
I20260808 22:31:38.544250 2253 tablet_bootstrap.cc:654] T d1aa1be2c714429d9deafd5f00abeeed P dba5657cda984625a81e9fa4dda06b8b: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:38.544250 2254 tablet_bootstrap.cc:654] T d1aa1be2c714429d9deafd5f00abeeed P 9f36d03373d54b269ac978d372329799: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:38.544291 2290 ts_tablet_manager.cc:1939] T c88c7b39e9134688afbf777e41aeaade P 9f36d03373d54b269ac978d372329799: Deleting consensus metadata
I20260808 22:31:38.545229 1359 catalog_manager.cc:4841] TS 9f36d03373d54b269ac978d372329799 (127.31.87.129:40357): tablet c88c7b39e9134688afbf777e41aeaade (table test-table [id=08a3e16adb0f408192b216a43a1e1126]) successfully deleted
I20260808 22:31:38.545620 2254 tablet_bootstrap.cc:492] T d1aa1be2c714429d9deafd5f00abeeed P 9f36d03373d54b269ac978d372329799: No bootstrap required, opened a new log
I20260808 22:31:38.545722 2254 ts_tablet_manager.cc:1397] T d1aa1be2c714429d9deafd5f00abeeed P 9f36d03373d54b269ac978d372329799: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 22:31:38.546115 2107 tablet_service.cc:1467] Processing CreateTablet for tablet d1aa1be2c714429d9deafd5f00abeeed (DEFAULT_TABLE table=test-table [id=6145aa535915492ba6da1a9576344bf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:31:38.546370 2107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1aa1be2c714429d9deafd5f00abeeed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:38.546404 2254 raft_consensus.cc:348] T d1aa1be2c714429d9deafd5f00abeeed P 9f36d03373d54b269ac978d372329799 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f36d03373d54b269ac978d372329799" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40357 } } peers { permanent_uuid: "dba5657cda984625a81e9fa4dda06b8b" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 44213 } } peers { permanent_uuid: "762affbd72ce479ca38322dc19ef2a80" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 38801 } }
I20260808 22:31:38.546507 2254 raft_consensus.cc:374] T d1aa1be2c714429d9deafd5f00abeeed P 9f36d03373d54b269ac978d372329799 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:38.546540 2254 raft_consensus.cc:729] T d1aa1be2c714429d9deafd5f00abeeed P 9f36d03373d54b269ac978d372329799 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f36d03373d54b269ac978d372329799, State: Initialized, Role: FOLLOWER
I20260808 22:31:38.546681 2254 consensus_queue.cc:260] T d1aa1be2c714429d9deafd5f00abeeed P 9f36d03373d54b269ac978d372329799 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f36d03373d54b269ac978d372329799" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40357 } } peers { permanent_uuid: "dba5657cda984625a81e9fa4dda06b8b" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 44213 } } peers { permanent_uuid: "762affbd72ce479ca38322dc19ef2a80" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 38801 } }
I20260808 22:31:38.547008 2253 tablet_bootstrap.cc:492] T d1aa1be2c714429d9deafd5f00abeeed P dba5657cda984625a81e9fa4dda06b8b: No bootstrap required, opened a new log
I20260808 22:31:38.547076 2253 ts_tablet_manager.cc:1397] T d1aa1be2c714429d9deafd5f00abeeed P dba5657cda984625a81e9fa4dda06b8b: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260808 22:31:38.547509 2253 raft_consensus.cc:348] T d1aa1be2c714429d9deafd5f00abeeed P dba5657cda984625a81e9fa4dda06b8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f36d03373d54b269ac978d372329799" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40357 } } peers { permanent_uuid: "dba5657cda984625a81e9fa4dda06b8b" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 44213 } } peers { permanent_uuid: "762affbd72ce479ca38322dc19ef2a80" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 38801 } }
I20260808 22:31:38.547603 2253 raft_consensus.cc:374] T d1aa1be2c714429d9deafd5f00abeeed P dba5657cda984625a81e9fa4dda06b8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:38.547659 2253 raft_consensus.cc:729] T d1aa1be2c714429d9deafd5f00abeeed P dba5657cda984625a81e9fa4dda06b8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dba5657cda984625a81e9fa4dda06b8b, State: Initialized, Role: FOLLOWER
I20260808 22:31:38.547772 2253 consensus_queue.cc:260] T d1aa1be2c714429d9deafd5f00abeeed P dba5657cda984625a81e9fa4dda06b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f36d03373d54b269ac978d372329799" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40357 } } peers { permanent_uuid: "dba5657cda984625a81e9fa4dda06b8b" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 44213 } } peers { permanent_uuid: "762affbd72ce479ca38322dc19ef2a80" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 38801 } }
I20260808 22:31:38.548020 2253 ts_tablet_manager.cc:1428] T d1aa1be2c714429d9deafd5f00abeeed P dba5657cda984625a81e9fa4dda06b8b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 22:31:38.548761 2256 tablet_bootstrap.cc:492] T d1aa1be2c714429d9deafd5f00abeeed P 762affbd72ce479ca38322dc19ef2a80: Bootstrap starting.
I20260808 22:31:38.549157 2254 ts_tablet_manager.cc:1428] T d1aa1be2c714429d9deafd5f00abeeed P 9f36d03373d54b269ac978d372329799: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260808 22:31:38.549824 2256 tablet_bootstrap.cc:654] T d1aa1be2c714429d9deafd5f00abeeed P 762affbd72ce479ca38322dc19ef2a80: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:38.551561 2256 tablet_bootstrap.cc:492] T d1aa1be2c714429d9deafd5f00abeeed P 762affbd72ce479ca38322dc19ef2a80: No bootstrap required, opened a new log
I20260808 22:31:38.551651 2256 ts_tablet_manager.cc:1397] T d1aa1be2c714429d9deafd5f00abeeed P 762affbd72ce479ca38322dc19ef2a80: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 22:31:38.552081 2256 raft_consensus.cc:348] T d1aa1be2c714429d9deafd5f00abeeed P 762affbd72ce479ca38322dc19ef2a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f36d03373d54b269ac978d372329799" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40357 } } peers { permanent_uuid: "dba5657cda984625a81e9fa4dda06b8b" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 44213 } } peers { permanent_uuid: "762affbd72ce479ca38322dc19ef2a80" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 38801 } }
I20260808 22:31:38.552192 2256 raft_consensus.cc:374] T d1aa1be2c714429d9deafd5f00abeeed P 762affbd72ce479ca38322dc19ef2a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:38.552222 2256 raft_consensus.cc:729] T d1aa1be2c714429d9deafd5f00abeeed P 762affbd72ce479ca38322dc19ef2a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 762affbd72ce479ca38322dc19ef2a80, State: Initialized, Role: FOLLOWER
I20260808 22:31:38.552345 2256 consensus_queue.cc:260] T d1aa1be2c714429d9deafd5f00abeeed P 762affbd72ce479ca38322dc19ef2a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f36d03373d54b269ac978d372329799" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40357 } } peers { permanent_uuid: "dba5657cda984625a81e9fa4dda06b8b" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 44213 } } peers { permanent_uuid: "762affbd72ce479ca38322dc19ef2a80" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 38801 } }
I20260808 22:31:38.552613 2256 ts_tablet_manager.cc:1428] T d1aa1be2c714429d9deafd5f00abeeed P 762affbd72ce479ca38322dc19ef2a80: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20260808 22:31:38.607623 2215 tablet.cc:2369] T d1aa1be2c714429d9deafd5f00abeeed P 762affbd72ce479ca38322dc19ef2a80: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 22:31:38.625411 1985 tablet.cc:2369] T d1aa1be2c714429d9deafd5f00abeeed P dba5657cda984625a81e9fa4dda06b8b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 22:31:38.651654 1745 tablet.cc:2369] T d1aa1be2c714429d9deafd5f00abeeed P 9f36d03373d54b269ac978d372329799: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:31:38.689816 2262 raft_consensus.cc:482] T d1aa1be2c714429d9deafd5f00abeeed P dba5657cda984625a81e9fa4dda06b8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:31:38.689990 2262 raft_consensus.cc:504] T d1aa1be2c714429d9deafd5f00abeeed P dba5657cda984625a81e9fa4dda06b8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f36d03373d54b269ac978d372329799" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40357 } } peers { permanent_uuid: "dba5657cda984625a81e9fa4dda06b8b" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 44213 } } peers { permanent_uuid: "762affbd72ce479ca38322dc19ef2a80" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 38801 } }
I20260808 22:31:38.691143 2262 leader_election.cc:288] T d1aa1be2c714429d9deafd5f00abeeed P dba5657cda984625a81e9fa4dda06b8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f36d03373d54b269ac978d372329799 (127.31.87.129:40357), 762affbd72ce479ca38322dc19ef2a80 (127.31.87.131:38801)
I20260808 22:31:38.695848 2252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:38.691519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:35207 (local address 127.31.87.129:40357)
0808 22:31:38.691652 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:38.691657 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:38.691806 (+ 149us) server_negotiation.cc:407] Connection header received
0808 22:31:38.691941 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:38.691944 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:38.692014 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:38.692081 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:31:38.692368 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:38.692961 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:38.694622 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:38.695205 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:38.695337 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:38.695383 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:38.695668 (+ 285us) server_negotiation.cc:299] Negotiation successful
0808 22:31:38.695731 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 22:31:38.695848 2251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:38.691501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:54783 (local address 127.31.87.131:38801)
0808 22:31:38.691660 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:38.691665 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:38.691893 (+ 228us) server_negotiation.cc:407] Connection header received
0808 22:31:38.692009 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:38.692012 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:38.692060 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:38.692126 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:31:38.692377 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:38.692940 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:38.694622 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:38.695216 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:38.695331 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:38.695383 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:38.695638 (+ 255us) server_negotiation.cc:299] Negotiation successful
0808 22:31:38.695713 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 22:31:38.695866 2295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:38.691390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.131:38801 (local address 127.31.87.130:54783)
0808 22:31:38.691811 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:38.691822 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:38.691919 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:38.692150 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:38.692153 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:38.692168 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:31:38.692281 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:38.692286 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:38.693039 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:38.693043 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:38.694499 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:38.694510 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:38.695316 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:38.695320 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:38.695493 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:38.695544 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 22:31:38.695619 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 22:31:38.695697 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"mutex_wait_us":113,"thread_start_us":74,"threads_started":1}
I20260808 22:31:38.696074 2296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:38.691377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:40357 (local address 127.31.87.130:35207)
0808 22:31:38.691719 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:38.691730 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:38.691844 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:38.692104 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:38.692107 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:38.692122 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:31:38.692260 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:38.692266 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:38.693066 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:38.693069 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:38.694499 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:38.694510 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:38.695320 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:38.695323 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:38.695493 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:38.695544 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 22:31:38.695656 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 22:31:38.695715 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"mutex_wait_us":178,"thread_start_us":86,"threads_started":1}
I20260808 22:31:38.696381 2144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1aa1be2c714429d9deafd5f00abeeed" candidate_uuid: "dba5657cda984625a81e9fa4dda06b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "762affbd72ce479ca38322dc19ef2a80" is_pre_election: true
I20260808 22:31:38.696570 1660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1aa1be2c714429d9deafd5f00abeeed" candidate_uuid: "dba5657cda984625a81e9fa4dda06b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f36d03373d54b269ac978d372329799" is_pre_election: true
I20260808 22:31:38.696640 2144 raft_consensus.cc:2454] T d1aa1be2c714429d9deafd5f00abeeed P 762affbd72ce479ca38322dc19ef2a80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dba5657cda984625a81e9fa4dda06b8b in term 0.
I20260808 22:31:38.696712 1660 raft_consensus.cc:2454] T d1aa1be2c714429d9deafd5f00abeeed P 9f36d03373d54b269ac978d372329799 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dba5657cda984625a81e9fa4dda06b8b in term 0.
I20260808 22:31:38.697044 1780 leader_election.cc:302] T d1aa1be2c714429d9deafd5f00abeeed P dba5657cda984625a81e9fa4dda06b8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 762affbd72ce479ca38322dc19ef2a80, dba5657cda984625a81e9fa4dda06b8b; no voters:
I20260808 22:31:38.697333 2262 raft_consensus.cc:2781] T d1aa1be2c714429d9deafd5f00abeeed P dba5657cda984625a81e9fa4dda06b8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:31:38.697396 2262 raft_consensus.cc:482] T d1aa1be2c714429d9deafd5f00abeeed P dba5657cda984625a81e9fa4dda06b8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:31:38.697439 2262 raft_consensus.cc:3037] T d1aa1be2c714429d9deafd5f00abeeed P dba5657cda984625a81e9fa4dda06b8b [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:38.698406 2262 raft_consensus.cc:504] T d1aa1be2c714429d9deafd5f00abeeed P dba5657cda984625a81e9fa4dda06b8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f36d03373d54b269ac978d372329799" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40357 } } peers { permanent_uuid: "dba5657cda984625a81e9fa4dda06b8b" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 44213 } } peers { permanent_uuid: "762affbd72ce479ca38322dc19ef2a80" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 38801 } }
I20260808 22:31:38.698707 2262 leader_election.cc:288] T d1aa1be2c714429d9deafd5f00abeeed P dba5657cda984625a81e9fa4dda06b8b [CANDIDATE]: Term 1 election: Requested vote from peers 9f36d03373d54b269ac978d372329799 (127.31.87.129:40357), 762affbd72ce479ca38322dc19ef2a80 (127.31.87.131:38801)
I20260808 22:31:38.699023 1660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1aa1be2c714429d9deafd5f00abeeed" candidate_uuid: "dba5657cda984625a81e9fa4dda06b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f36d03373d54b269ac978d372329799"
I20260808 22:31:38.699064 2144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1aa1be2c714429d9deafd5f00abeeed" candidate_uuid: "dba5657cda984625a81e9fa4dda06b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "762affbd72ce479ca38322dc19ef2a80"
I20260808 22:31:38.699162 2144 raft_consensus.cc:3037] T d1aa1be2c714429d9deafd5f00abeeed P 762affbd72ce479ca38322dc19ef2a80 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:38.699162 1660 raft_consensus.cc:3037] T d1aa1be2c714429d9deafd5f00abeeed P 9f36d03373d54b269ac978d372329799 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:38.700254 1660 raft_consensus.cc:2454] T d1aa1be2c714429d9deafd5f00abeeed P 9f36d03373d54b269ac978d372329799 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dba5657cda984625a81e9fa4dda06b8b in term 1.
I20260808 22:31:38.700254 2144 raft_consensus.cc:2454] T d1aa1be2c714429d9deafd5f00abeeed P 762affbd72ce479ca38322dc19ef2a80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dba5657cda984625a81e9fa4dda06b8b in term 1.
I20260808 22:31:38.700526 1776 leader_election.cc:302] T d1aa1be2c714429d9deafd5f00abeeed P dba5657cda984625a81e9fa4dda06b8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f36d03373d54b269ac978d372329799, dba5657cda984625a81e9fa4dda06b8b; no voters:
I20260808 22:31:38.700682 2262 raft_consensus.cc:2781] T d1aa1be2c714429d9deafd5f00abeeed P dba5657cda984625a81e9fa4dda06b8b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:31:38.700858 2262 raft_consensus.cc:686] T d1aa1be2c714429d9deafd5f00abeeed P dba5657cda984625a81e9fa4dda06b8b [term 1 LEADER]: Becoming Leader. State: Replica: dba5657cda984625a81e9fa4dda06b8b, State: Running, Role: LEADER
I20260808 22:31:38.701082 2262 consensus_queue.cc:237] T d1aa1be2c714429d9deafd5f00abeeed P dba5657cda984625a81e9fa4dda06b8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f36d03373d54b269ac978d372329799" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40357 } } peers { permanent_uuid: "dba5657cda984625a81e9fa4dda06b8b" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 44213 } } peers { permanent_uuid: "762affbd72ce479ca38322dc19ef2a80" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 38801 } }
I20260808 22:31:38.703329 1378 catalog_manager.cc:5495] T d1aa1be2c714429d9deafd5f00abeeed P dba5657cda984625a81e9fa4dda06b8b reported cstate change: term changed from 0 to 1, leader changed from <none> to dba5657cda984625a81e9fa4dda06b8b (127.31.87.130). New cstate: current_term: 1 leader_uuid: "dba5657cda984625a81e9fa4dda06b8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f36d03373d54b269ac978d372329799" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dba5657cda984625a81e9fa4dda06b8b" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 44213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "762affbd72ce479ca38322dc19ef2a80" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 38801 } health_report { overall_health: UNKNOWN } } }
Aug 08 22:31:38 dist-test-slave-266d krb5kdc[1318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228297, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:31:38.758126 2313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:38.750593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:39551 (local address 127.0.0.1:58024)
0808 22:31:38.750865 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:38.750880 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:38.750977 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:38.751181 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:38.751183 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:38.751576 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:38.751669 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:38.751675 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:38.752392 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:38.752394 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:38.752935 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:38.752941 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:38.753376 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:38.753379 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:38.753441 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:38.754010 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:31:38.754022 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:38.755404 (+ 1382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:38.756790 (+ 1386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:38.756799 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:38.756805 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:38.757024 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:38.757266 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:38.757269 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:38.757271 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:38.757343 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:38.757654 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:38.757659 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:38.757760 (+ 101us) client_negotiation.cc:769] Sending connection context
0808 22:31:38.757854 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 22:31:38.757970 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":105,"threads_started":1}
I20260808 22:31:38.758153 1472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:38.750632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58024 (local address 127.31.87.190:39551)
0808 22:31:38.750734 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:38.750737 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:38.750940 (+ 203us) server_negotiation.cc:407] Connection header received
0808 22:31:38.751059 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:38.751061 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:38.751100 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:38.751158 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:38.751786 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:38.752310 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:38.753014 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:38.753284 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:38.753359 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:38.755526 (+ 2167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:38.755529 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:38.755531 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:38.755552 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:38.756694 (+ 1142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:38.757119 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:38.757123 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:38.757125 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:38.757197 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:38.757404 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:38.757406 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:38.757407 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:38.757560 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:38.757657 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:38.757895 (+ 238us) server_negotiation.cc:299] Negotiation successful
0808 22:31:38.758023 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":33}
I20260808 22:31:38.761956 1378 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:58024:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 22:31:38.762677 1378 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:31:38.767232 1863 tablet_service.cc:1467] Processing CreateTablet for tablet 4321c79e6fb44301b9c778f0588d8ee2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d9b42ffd625f4527bc7fdd8f7482e91f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 22:31:38.767253 1631 tablet_service.cc:1467] Processing CreateTablet for tablet 4321c79e6fb44301b9c778f0588d8ee2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d9b42ffd625f4527bc7fdd8f7482e91f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 22:31:38.767309 2107 tablet_service.cc:1467] Processing CreateTablet for tablet 4321c79e6fb44301b9c778f0588d8ee2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d9b42ffd625f4527bc7fdd8f7482e91f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 22:31:38.767648 2107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4321c79e6fb44301b9c778f0588d8ee2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:38.767647 1631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4321c79e6fb44301b9c778f0588d8ee2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:38.767652 1863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4321c79e6fb44301b9c778f0588d8ee2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:38.770128 2253 tablet_bootstrap.cc:492] T 4321c79e6fb44301b9c778f0588d8ee2 P dba5657cda984625a81e9fa4dda06b8b: Bootstrap starting.
I20260808 22:31:38.770325 2256 tablet_bootstrap.cc:492] T 4321c79e6fb44301b9c778f0588d8ee2 P 762affbd72ce479ca38322dc19ef2a80: Bootstrap starting.
I20260808 22:31:38.771030 2253 tablet_bootstrap.cc:654] T 4321c79e6fb44301b9c778f0588d8ee2 P dba5657cda984625a81e9fa4dda06b8b: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:38.771044 2256 tablet_bootstrap.cc:654] T 4321c79e6fb44301b9c778f0588d8ee2 P 762affbd72ce479ca38322dc19ef2a80: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:38.771716 2254 tablet_bootstrap.cc:492] T 4321c79e6fb44301b9c778f0588d8ee2 P 9f36d03373d54b269ac978d372329799: Bootstrap starting.
I20260808 22:31:38.772037 2256 tablet_bootstrap.cc:492] T 4321c79e6fb44301b9c778f0588d8ee2 P 762affbd72ce479ca38322dc19ef2a80: No bootstrap required, opened a new log
I20260808 22:31:38.772125 2256 ts_tablet_manager.cc:1397] T 4321c79e6fb44301b9c778f0588d8ee2 P 762affbd72ce479ca38322dc19ef2a80: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 22:31:38.772421 2253 tablet_bootstrap.cc:492] T 4321c79e6fb44301b9c778f0588d8ee2 P dba5657cda984625a81e9fa4dda06b8b: No bootstrap required, opened a new log
I20260808 22:31:38.772507 2253 ts_tablet_manager.cc:1397] T 4321c79e6fb44301b9c778f0588d8ee2 P dba5657cda984625a81e9fa4dda06b8b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 22:31:38.772547 2256 raft_consensus.cc:348] T 4321c79e6fb44301b9c778f0588d8ee2 P 762affbd72ce479ca38322dc19ef2a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f36d03373d54b269ac978d372329799" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40357 } } peers { permanent_uuid: "dba5657cda984625a81e9fa4dda06b8b" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 44213 } } peers { permanent_uuid: "762affbd72ce479ca38322dc19ef2a80" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 38801 } }
I20260808 22:31:38.772648 2254 tablet_bootstrap.cc:654] T 4321c79e6fb44301b9c778f0588d8ee2 P 9f36d03373d54b269ac978d372329799: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:38.772662 2256 raft_consensus.cc:374] T 4321c79e6fb44301b9c778f0588d8ee2 P 762affbd72ce479ca38322dc19ef2a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:38.772699 2256 raft_consensus.cc:729] T 4321c79e6fb44301b9c778f0588d8ee2 P 762affbd72ce479ca38322dc19ef2a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 762affbd72ce479ca38322dc19ef2a80, State: Initialized, Role: FOLLOWER
I20260808 22:31:38.772833 2256 consensus_queue.cc:260] T 4321c79e6fb44301b9c778f0588d8ee2 P 762affbd72ce479ca38322dc19ef2a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f36d03373d54b269ac978d372329799" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40357 } } peers { permanent_uuid: "dba5657cda984625a81e9fa4dda06b8b" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 44213 } } peers { permanent_uuid: "762affbd72ce479ca38322dc19ef2a80" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 38801 } }
I20260808 22:31:38.772944 2253 raft_consensus.cc:348] T 4321c79e6fb44301b9c778f0588d8ee2 P dba5657cda984625a81e9fa4dda06b8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f36d03373d54b269ac978d372329799" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40357 } } peers { permanent_uuid: "dba5657cda984625a81e9fa4dda06b8b" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 44213 } } peers { permanent_uuid: "762affbd72ce479ca38322dc19ef2a80" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 38801 } }
I20260808 22:31:38.773043 2253 raft_consensus.cc:374] T 4321c79e6fb44301b9c778f0588d8ee2 P dba5657cda984625a81e9fa4dda06b8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:38.773080 2253 raft_consensus.cc:729] T 4321c79e6fb44301b9c778f0588d8ee2 P dba5657cda984625a81e9fa4dda06b8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dba5657cda984625a81e9fa4dda06b8b, State: Initialized, Role: FOLLOWER
I20260808 22:31:38.773237 2253 consensus_queue.cc:260] T 4321c79e6fb44301b9c778f0588d8ee2 P dba5657cda984625a81e9fa4dda06b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f36d03373d54b269ac978d372329799" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40357 } } peers { permanent_uuid: "dba5657cda984625a81e9fa4dda06b8b" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 44213 } } peers { permanent_uuid: "762affbd72ce479ca38322dc19ef2a80" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 38801 } }
I20260808 22:31:38.773314 2269 tablet_replica.cc:439] T 4321c79e6fb44301b9c778f0588d8ee2 P 762affbd72ce479ca38322dc19ef2a80: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f36d03373d54b269ac978d372329799" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40357 } } peers { permanent_uuid: "dba5657cda984625a81e9fa4dda06b8b" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 44213 } } peers { permanent_uuid: "762affbd72ce479ca38322dc19ef2a80" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 38801 } } }
I20260808 22:31:38.773442 2269 tablet_replica.cc:442] T 4321c79e6fb44301b9c778f0588d8ee2 P 762affbd72ce479ca38322dc19ef2a80: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:31:38.773518 2253 ts_tablet_manager.cc:1428] T 4321c79e6fb44301b9c778f0588d8ee2 P dba5657cda984625a81e9fa4dda06b8b: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 22:31:38.773705 2256 ts_tablet_manager.cc:1428] T 4321c79e6fb44301b9c778f0588d8ee2 P 762affbd72ce479ca38322dc19ef2a80: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260808 22:31:38.774085 2262 tablet_replica.cc:439] T 4321c79e6fb44301b9c778f0588d8ee2 P dba5657cda984625a81e9fa4dda06b8b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f36d03373d54b269ac978d372329799" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40357 } } peers { permanent_uuid: "dba5657cda984625a81e9fa4dda06b8b" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 44213 } } peers { permanent_uuid: "762affbd72ce479ca38322dc19ef2a80" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 38801 } } }
I20260808 22:31:38.774210 2262 tablet_replica.cc:442] T 4321c79e6fb44301b9c778f0588d8ee2 P dba5657cda984625a81e9fa4dda06b8b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:31:38.774474 2254 tablet_bootstrap.cc:492] T 4321c79e6fb44301b9c778f0588d8ee2 P 9f36d03373d54b269ac978d372329799: No bootstrap required, opened a new log
I20260808 22:31:38.774556 2254 ts_tablet_manager.cc:1397] T 4321c79e6fb44301b9c778f0588d8ee2 P 9f36d03373d54b269ac978d372329799: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260808 22:31:38.774986 2254 raft_consensus.cc:348] T 4321c79e6fb44301b9c778f0588d8ee2 P 9f36d03373d54b269ac978d372329799 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f36d03373d54b269ac978d372329799" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40357 } } peers { permanent_uuid: "dba5657cda984625a81e9fa4dda06b8b" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 44213 } } peers { permanent_uuid: "762affbd72ce479ca38322dc19ef2a80" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 38801 } }
I20260808 22:31:38.775090 2254 raft_consensus.cc:374] T 4321c79e6fb44301b9c778f0588d8ee2 P 9f36d03373d54b269ac978d372329799 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:38.775120 2254 raft_consensus.cc:729] T 4321c79e6fb44301b9c778f0588d8ee2 P 9f36d03373d54b269ac978d372329799 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f36d03373d54b269ac978d372329799, State: Initialized, Role: FOLLOWER
I20260808 22:31:38.775224 2254 consensus_queue.cc:260] T 4321c79e6fb44301b9c778f0588d8ee2 P 9f36d03373d54b269ac978d372329799 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f36d03373d54b269ac978d372329799" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40357 } } peers { permanent_uuid: "dba5657cda984625a81e9fa4dda06b8b" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 44213 } } peers { permanent_uuid: "762affbd72ce479ca38322dc19ef2a80" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 38801 } }
I20260808 22:31:38.775460 2254 ts_tablet_manager.cc:1428] T 4321c79e6fb44301b9c778f0588d8ee2 P 9f36d03373d54b269ac978d372329799: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 22:31:38.775465 2265 tablet_replica.cc:439] T 4321c79e6fb44301b9c778f0588d8ee2 P 9f36d03373d54b269ac978d372329799: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f36d03373d54b269ac978d372329799" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40357 } } peers { permanent_uuid: "dba5657cda984625a81e9fa4dda06b8b" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 44213 } } peers { permanent_uuid: "762affbd72ce479ca38322dc19ef2a80" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 38801 } } }
I20260808 22:31:38.775691 2265 tablet_replica.cc:442] T 4321c79e6fb44301b9c778f0588d8ee2 P 9f36d03373d54b269ac978d372329799: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:31:39.113859 2262 consensus_queue.cc:1035] T d1aa1be2c714429d9deafd5f00abeeed P dba5657cda984625a81e9fa4dda06b8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f36d03373d54b269ac978d372329799" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:31:39.117791 2327 consensus_queue.cc:1035] T d1aa1be2c714429d9deafd5f00abeeed P dba5657cda984625a81e9fa4dda06b8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "762affbd72ce479ca38322dc19ef2a80" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 38801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:31:39.189632 2327 raft_consensus.cc:482] T 4321c79e6fb44301b9c778f0588d8ee2 P dba5657cda984625a81e9fa4dda06b8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:31:39.189771 2327 raft_consensus.cc:504] T 4321c79e6fb44301b9c778f0588d8ee2 P dba5657cda984625a81e9fa4dda06b8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f36d03373d54b269ac978d372329799" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40357 } } peers { permanent_uuid: "dba5657cda984625a81e9fa4dda06b8b" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 44213 } } peers { permanent_uuid: "762affbd72ce479ca38322dc19ef2a80" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 38801 } }
I20260808 22:31:39.190209 2327 leader_election.cc:288] T 4321c79e6fb44301b9c778f0588d8ee2 P dba5657cda984625a81e9fa4dda06b8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f36d03373d54b269ac978d372329799 (127.31.87.129:40357), 762affbd72ce479ca38322dc19ef2a80 (127.31.87.131:38801)
I20260808 22:31:39.190699 2144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4321c79e6fb44301b9c778f0588d8ee2" candidate_uuid: "dba5657cda984625a81e9fa4dda06b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "762affbd72ce479ca38322dc19ef2a80" is_pre_election: true
I20260808 22:31:39.190707 1660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4321c79e6fb44301b9c778f0588d8ee2" candidate_uuid: "dba5657cda984625a81e9fa4dda06b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f36d03373d54b269ac978d372329799" is_pre_election: true
I20260808 22:31:39.190860 1660 raft_consensus.cc:2454] T 4321c79e6fb44301b9c778f0588d8ee2 P 9f36d03373d54b269ac978d372329799 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dba5657cda984625a81e9fa4dda06b8b in term 0.
I20260808 22:31:39.190850 2144 raft_consensus.cc:2454] T 4321c79e6fb44301b9c778f0588d8ee2 P 762affbd72ce479ca38322dc19ef2a80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dba5657cda984625a81e9fa4dda06b8b in term 0.
I20260808 22:31:39.191177 1780 leader_election.cc:302] T 4321c79e6fb44301b9c778f0588d8ee2 P dba5657cda984625a81e9fa4dda06b8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 762affbd72ce479ca38322dc19ef2a80, dba5657cda984625a81e9fa4dda06b8b; no voters:
I20260808 22:31:39.191345 2327 raft_consensus.cc:2781] T 4321c79e6fb44301b9c778f0588d8ee2 P dba5657cda984625a81e9fa4dda06b8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:31:39.191421 2327 raft_consensus.cc:482] T 4321c79e6fb44301b9c778f0588d8ee2 P dba5657cda984625a81e9fa4dda06b8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:31:39.191465 2327 raft_consensus.cc:3037] T 4321c79e6fb44301b9c778f0588d8ee2 P dba5657cda984625a81e9fa4dda06b8b [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:39.192552 2327 raft_consensus.cc:504] T 4321c79e6fb44301b9c778f0588d8ee2 P dba5657cda984625a81e9fa4dda06b8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f36d03373d54b269ac978d372329799" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40357 } } peers { permanent_uuid: "dba5657cda984625a81e9fa4dda06b8b" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 44213 } } peers { permanent_uuid: "762affbd72ce479ca38322dc19ef2a80" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 38801 } }
I20260808 22:31:39.192909 2327 leader_election.cc:288] T 4321c79e6fb44301b9c778f0588d8ee2 P dba5657cda984625a81e9fa4dda06b8b [CANDIDATE]: Term 1 election: Requested vote from peers 9f36d03373d54b269ac978d372329799 (127.31.87.129:40357), 762affbd72ce479ca38322dc19ef2a80 (127.31.87.131:38801)
I20260808 22:31:39.193240 1660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4321c79e6fb44301b9c778f0588d8ee2" candidate_uuid: "dba5657cda984625a81e9fa4dda06b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f36d03373d54b269ac978d372329799"
I20260808 22:31:39.193256 2144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4321c79e6fb44301b9c778f0588d8ee2" candidate_uuid: "dba5657cda984625a81e9fa4dda06b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "762affbd72ce479ca38322dc19ef2a80"
I20260808 22:31:39.193342 2144 raft_consensus.cc:3037] T 4321c79e6fb44301b9c778f0588d8ee2 P 762affbd72ce479ca38322dc19ef2a80 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:39.193349 1660 raft_consensus.cc:3037] T 4321c79e6fb44301b9c778f0588d8ee2 P 9f36d03373d54b269ac978d372329799 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:39.194205 2144 raft_consensus.cc:2454] T 4321c79e6fb44301b9c778f0588d8ee2 P 762affbd72ce479ca38322dc19ef2a80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dba5657cda984625a81e9fa4dda06b8b in term 1.
I20260808 22:31:39.194212 1660 raft_consensus.cc:2454] T 4321c79e6fb44301b9c778f0588d8ee2 P 9f36d03373d54b269ac978d372329799 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dba5657cda984625a81e9fa4dda06b8b in term 1.
I20260808 22:31:39.194447 1776 leader_election.cc:302] T 4321c79e6fb44301b9c778f0588d8ee2 P dba5657cda984625a81e9fa4dda06b8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f36d03373d54b269ac978d372329799, dba5657cda984625a81e9fa4dda06b8b; no voters:
I20260808 22:31:39.194597 2327 raft_consensus.cc:2781] T 4321c79e6fb44301b9c778f0588d8ee2 P dba5657cda984625a81e9fa4dda06b8b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:31:39.194682 2327 raft_consensus.cc:686] T 4321c79e6fb44301b9c778f0588d8ee2 P dba5657cda984625a81e9fa4dda06b8b [term 1 LEADER]: Becoming Leader. State: Replica: dba5657cda984625a81e9fa4dda06b8b, State: Running, Role: LEADER
I20260808 22:31:39.194815 2327 consensus_queue.cc:237] T 4321c79e6fb44301b9c778f0588d8ee2 P dba5657cda984625a81e9fa4dda06b8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f36d03373d54b269ac978d372329799" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40357 } } peers { permanent_uuid: "dba5657cda984625a81e9fa4dda06b8b" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 44213 } } peers { permanent_uuid: "762affbd72ce479ca38322dc19ef2a80" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 38801 } }
I20260808 22:31:39.195412 2262 tablet_replica.cc:439] T 4321c79e6fb44301b9c778f0588d8ee2 P dba5657cda984625a81e9fa4dda06b8b: TxnStatusTablet state changed. Reason: New leader dba5657cda984625a81e9fa4dda06b8b. Latest consensus state: current_term: 1 leader_uuid: "dba5657cda984625a81e9fa4dda06b8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f36d03373d54b269ac978d372329799" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40357 } } peers { permanent_uuid: "dba5657cda984625a81e9fa4dda06b8b" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 44213 } } peers { permanent_uuid: "762affbd72ce479ca38322dc19ef2a80" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 38801 } } }
I20260808 22:31:39.195518 2262 tablet_replica.cc:442] T 4321c79e6fb44301b9c778f0588d8ee2 P dba5657cda984625a81e9fa4dda06b8b: This TxnStatusTablet replica's current role is: LEADER
I20260808 22:31:39.195820 2337 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 22:31:39.196043 2337 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 22:31:39.196493 1376 catalog_manager.cc:5495] T 4321c79e6fb44301b9c778f0588d8ee2 P dba5657cda984625a81e9fa4dda06b8b reported cstate change: term changed from 0 to 1, leader changed from <none> to dba5657cda984625a81e9fa4dda06b8b (127.31.87.130). New cstate: current_term: 1 leader_uuid: "dba5657cda984625a81e9fa4dda06b8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f36d03373d54b269ac978d372329799" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dba5657cda984625a81e9fa4dda06b8b" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 44213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "762affbd72ce479ca38322dc19ef2a80" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 38801 } health_report { overall_health: UNKNOWN } } }
I20260808 22:31:39.226958 2342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:39.223528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33176 (local address 127.31.87.130:44213)
0808 22:31:39.223750 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:39.223755 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:39.223766 (+ 11us) server_negotiation.cc:407] Connection header received
0808 22:31:39.223822 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:39.223824 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:39.223870 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:39.223955 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:31:39.224201 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:39.224730 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:39.225564 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:39.225803 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:39.225890 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:39.226188 (+ 298us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:31:39.226567 (+ 379us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:31:39.226664 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:39.226748 (+ 84us) server_negotiation.cc:299] Negotiation successful
0808 22:31:39.226801 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":70,"threads_started":1}
I20260808 22:31:39.226944 2313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:39.223408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.130:44213 (local address 127.0.0.1:33176)
0808 22:31:39.223553 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:39.223568 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:39.223659 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:39.223981 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:39.223984 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:39.223991 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 22:31:39.224092 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:39.224098 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:39.224829 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:39.224833 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:39.225454 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:39.225462 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:39.225900 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:39.225904 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:39.225987 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:39.226032 (+ 45us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 22:31:39.226658 (+ 626us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 22:31:39.226674 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 22:31:39.226738 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 22:31:39.226833 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 22:31:39.229408 1660 raft_consensus.cc:1261] T 4321c79e6fb44301b9c778f0588d8ee2 P 9f36d03373d54b269ac978d372329799 [term 1 FOLLOWER]: Refusing update from remote peer dba5657cda984625a81e9fa4dda06b8b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:31:39.229432 2144 raft_consensus.cc:1261] T 4321c79e6fb44301b9c778f0588d8ee2 P 762affbd72ce479ca38322dc19ef2a80 [term 1 FOLLOWER]: Refusing update from remote peer dba5657cda984625a81e9fa4dda06b8b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:31:39.229789 2262 consensus_queue.cc:1035] T 4321c79e6fb44301b9c778f0588d8ee2 P dba5657cda984625a81e9fa4dda06b8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f36d03373d54b269ac978d372329799" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:31:39.229979 2327 consensus_queue.cc:1035] T 4321c79e6fb44301b9c778f0588d8ee2 P dba5657cda984625a81e9fa4dda06b8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "762affbd72ce479ca38322dc19ef2a80" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 38801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:31:39.231123 2265 tablet_replica.cc:439] T 4321c79e6fb44301b9c778f0588d8ee2 P 9f36d03373d54b269ac978d372329799: TxnStatusTablet state changed. Reason: New leader dba5657cda984625a81e9fa4dda06b8b. Latest consensus state: current_term: 1 leader_uuid: "dba5657cda984625a81e9fa4dda06b8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f36d03373d54b269ac978d372329799" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40357 } } peers { permanent_uuid: "dba5657cda984625a81e9fa4dda06b8b" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 44213 } } peers { permanent_uuid: "762affbd72ce479ca38322dc19ef2a80" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 38801 } } }
I20260808 22:31:39.231242 2265 tablet_replica.cc:442] T 4321c79e6fb44301b9c778f0588d8ee2 P 9f36d03373d54b269ac978d372329799: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:31:39.231653 2269 tablet_replica.cc:439] T 4321c79e6fb44301b9c778f0588d8ee2 P 762affbd72ce479ca38322dc19ef2a80: TxnStatusTablet state changed. Reason: New leader dba5657cda984625a81e9fa4dda06b8b. Latest consensus state: current_term: 1 leader_uuid: "dba5657cda984625a81e9fa4dda06b8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f36d03373d54b269ac978d372329799" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40357 } } peers { permanent_uuid: "dba5657cda984625a81e9fa4dda06b8b" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 44213 } } peers { permanent_uuid: "762affbd72ce479ca38322dc19ef2a80" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 38801 } } }
I20260808 22:31:39.231757 2269 tablet_replica.cc:442] T 4321c79e6fb44301b9c778f0588d8ee2 P 762affbd72ce479ca38322dc19ef2a80: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:31:39.232758 2327 tablet_replica.cc:439] T 4321c79e6fb44301b9c778f0588d8ee2 P dba5657cda984625a81e9fa4dda06b8b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dba5657cda984625a81e9fa4dda06b8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f36d03373d54b269ac978d372329799" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40357 } } peers { permanent_uuid: "dba5657cda984625a81e9fa4dda06b8b" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 44213 } } peers { permanent_uuid: "762affbd72ce479ca38322dc19ef2a80" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 38801 } } }
I20260808 22:31:39.232858 2327 tablet_replica.cc:442] T 4321c79e6fb44301b9c778f0588d8ee2 P dba5657cda984625a81e9fa4dda06b8b: This TxnStatusTablet replica's current role is: LEADER
I20260808 22:31:39.233122 2329 mvcc.cc:205] Tried to move back new op lower bound from 7316391113640026112 to 7316391113503166464. Current Snapshot: MvccSnapshot[applied={T|T < 7316391113640026112}]
I20260808 22:31:39.233228 2265 tablet_replica.cc:439] T 4321c79e6fb44301b9c778f0588d8ee2 P 9f36d03373d54b269ac978d372329799: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dba5657cda984625a81e9fa4dda06b8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f36d03373d54b269ac978d372329799" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40357 } } peers { permanent_uuid: "dba5657cda984625a81e9fa4dda06b8b" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 44213 } } peers { permanent_uuid: "762affbd72ce479ca38322dc19ef2a80" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 38801 } } }
I20260808 22:31:39.233321 2265 tablet_replica.cc:442] T 4321c79e6fb44301b9c778f0588d8ee2 P 9f36d03373d54b269ac978d372329799: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:31:39.233659 2325 mvcc.cc:205] Tried to move back new op lower bound from 7316391113640026112 to 7316391113503166464. Current Snapshot: MvccSnapshot[applied={T|T < 7316391113640026112}]
I20260808 22:31:39.237296 2297 tablet_replica.cc:439] T 4321c79e6fb44301b9c778f0588d8ee2 P dba5657cda984625a81e9fa4dda06b8b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dba5657cda984625a81e9fa4dda06b8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f36d03373d54b269ac978d372329799" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40357 } } peers { permanent_uuid: "dba5657cda984625a81e9fa4dda06b8b" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 44213 } } peers { permanent_uuid: "762affbd72ce479ca38322dc19ef2a80" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 38801 } } }
I20260808 22:31:39.237411 2297 tablet_replica.cc:442] T 4321c79e6fb44301b9c778f0588d8ee2 P dba5657cda984625a81e9fa4dda06b8b: This TxnStatusTablet replica's current role is: LEADER
I20260808 22:31:39.237972 2269 tablet_replica.cc:439] T 4321c79e6fb44301b9c778f0588d8ee2 P 762affbd72ce479ca38322dc19ef2a80: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dba5657cda984625a81e9fa4dda06b8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f36d03373d54b269ac978d372329799" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40357 } } peers { permanent_uuid: "dba5657cda984625a81e9fa4dda06b8b" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 44213 } } peers { permanent_uuid: "762affbd72ce479ca38322dc19ef2a80" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 38801 } } }
I20260808 22:31:39.238085 2269 tablet_replica.cc:442] T 4321c79e6fb44301b9c778f0588d8ee2 P 762affbd72ce479ca38322dc19ef2a80: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:31:39.243709 2342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:39.240589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33178 (local address 127.31.87.130:44213)
0808 22:31:39.240694 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:39.240698 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:39.240827 (+ 129us) server_negotiation.cc:407] Connection header received
0808 22:31:39.240943 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:39.240945 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:39.240981 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:39.241038 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:31:39.241293 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:39.241758 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:39.242897 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:39.243087 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:39.243149 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:39.243380 (+ 231us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:31:39.243437 (+ 57us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:31:39.243481 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:39.243572 (+ 91us) server_negotiation.cc:299] Negotiation successful
0808 22:31:39.243609 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 22:31:39.253166 2342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:39.250221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:40939 (local address 127.31.87.130:44213)
0808 22:31:39.250323 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:39.250327 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:39.250546 (+ 219us) server_negotiation.cc:407] Connection header received
0808 22:31:39.250644 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:39.250647 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:39.250683 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:39.250736 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:31:39.250960 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:39.251425 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:39.252212 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:39.252411 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:39.252491 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:39.252728 (+ 237us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:31:39.252786 (+ 58us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:31:39.252842 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:39.252922 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 22:31:39.253013 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":29}
I20260808 22:31:39.253201 2356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:39.250165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.130:44213 (local address 127.31.87.130:40939)
0808 22:31:39.250473 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:39.250484 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:39.250575 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:39.250749 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:39.250751 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:39.250757 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 22:31:39.250876 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:39.250881 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:39.251515 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:39.251518 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:39.252112 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:39.252120 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:39.252497 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:39.252500 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:39.252582 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:39.252621 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 22:31:39.252853 (+ 232us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 22:31:39.252860 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 22:31:39.252915 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 22:31:39.253039 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"mutex_wait_us":30,"thread_start_us":93,"threads_started":1}
I20260808 22:31:39.295361 1374 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:31:39.295497 1374 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:31:39.297323 1374 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 989bdf9e19a44aa3987414387310f4b9: Sending DeleteTablet for 3 replicas of tablet d1aa1be2c714429d9deafd5f00abeeed
I20260808 22:31:39.297851 1631 tablet_service.cc:1514] Processing DeleteTablet for tablet d1aa1be2c714429d9deafd5f00abeeed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:31:39 UTC) from {username='kudu', principal='kudu/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:48938
I20260808 22:31:39.297849 1863 tablet_service.cc:1514] Processing DeleteTablet for tablet d1aa1be2c714429d9deafd5f00abeeed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:31:39 UTC) from {username='kudu', principal='kudu/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:33162
I20260808 22:31:39.298038 2107 tablet_service.cc:1514] Processing DeleteTablet for tablet d1aa1be2c714429d9deafd5f00abeeed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:31:39 UTC) from {username='kudu', principal='kudu/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:48584
I20260808 22:31:39.298301 2360 tablet_replica.cc:330] T d1aa1be2c714429d9deafd5f00abeeed P 762affbd72ce479ca38322dc19ef2a80: stopping tablet replica
I20260808 22:31:39.298303 2358 tablet_replica.cc:330] T d1aa1be2c714429d9deafd5f00abeeed P 9f36d03373d54b269ac978d372329799: stopping tablet replica
I20260808 22:31:39.298422 2360 raft_consensus.cc:2229] T d1aa1be2c714429d9deafd5f00abeeed P 762affbd72ce479ca38322dc19ef2a80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:31:39.298524 2360 raft_consensus.cc:2258] T d1aa1be2c714429d9deafd5f00abeeed P 762affbd72ce479ca38322dc19ef2a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:31:39.298512 2358 raft_consensus.cc:2229] T d1aa1be2c714429d9deafd5f00abeeed P 9f36d03373d54b269ac978d372329799 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:31:39.298673 2358 raft_consensus.cc:2258] T d1aa1be2c714429d9deafd5f00abeeed P 9f36d03373d54b269ac978d372329799 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:31:39.299152 2360 ts_tablet_manager.cc:1905] T d1aa1be2c714429d9deafd5f00abeeed P 762affbd72ce479ca38322dc19ef2a80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:31:39.299208 2359 tablet_replica.cc:330] T d1aa1be2c714429d9deafd5f00abeeed P dba5657cda984625a81e9fa4dda06b8b: stopping tablet replica
I20260808 22:31:39.299320 2359 raft_consensus.cc:2229] T d1aa1be2c714429d9deafd5f00abeeed P dba5657cda984625a81e9fa4dda06b8b [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:31:39.299484 2358 ts_tablet_manager.cc:1905] T d1aa1be2c714429d9deafd5f00abeeed P 9f36d03373d54b269ac978d372329799: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:31:39.299584 2359 raft_consensus.cc:2258] T d1aa1be2c714429d9deafd5f00abeeed P dba5657cda984625a81e9fa4dda06b8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:31:39.300206 2359 ts_tablet_manager.cc:1905] T d1aa1be2c714429d9deafd5f00abeeed P dba5657cda984625a81e9fa4dda06b8b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:31:39.302086 2360 ts_tablet_manager.cc:1918] T d1aa1be2c714429d9deafd5f00abeeed P 762affbd72ce479ca38322dc19ef2a80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 22:31:39.302172 2360 log.cc:1196] T d1aa1be2c714429d9deafd5f00abeeed P 762affbd72ce479ca38322dc19ef2a80: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-2/wal/wals/d1aa1be2c714429d9deafd5f00abeeed
I20260808 22:31:39.302423 2360 ts_tablet_manager.cc:1939] T d1aa1be2c714429d9deafd5f00abeeed P 762affbd72ce479ca38322dc19ef2a80: Deleting consensus metadata
I20260808 22:31:39.302407 2358 ts_tablet_manager.cc:1918] T d1aa1be2c714429d9deafd5f00abeeed P 9f36d03373d54b269ac978d372329799: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 22:31:39.302560 2358 log.cc:1196] T d1aa1be2c714429d9deafd5f00abeeed P 9f36d03373d54b269ac978d372329799: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-0/wal/wals/d1aa1be2c714429d9deafd5f00abeeed
I20260808 22:31:39.302855 2358 ts_tablet_manager.cc:1939] T d1aa1be2c714429d9deafd5f00abeeed P 9f36d03373d54b269ac978d372329799: Deleting consensus metadata
I20260808 22:31:39.303079 1362 catalog_manager.cc:4841] TS 762affbd72ce479ca38322dc19ef2a80 (127.31.87.131:38801): tablet d1aa1be2c714429d9deafd5f00abeeed (table test-table [id=6145aa535915492ba6da1a9576344bf0]) successfully deleted
I20260808 22:31:39.303128 2359 ts_tablet_manager.cc:1918] T d1aa1be2c714429d9deafd5f00abeeed P dba5657cda984625a81e9fa4dda06b8b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 22:31:39.303192 2359 log.cc:1196] T d1aa1be2c714429d9deafd5f00abeeed P dba5657cda984625a81e9fa4dda06b8b: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786228291013016-32094-0/minicluster-data/ts-1/wal/wals/d1aa1be2c714429d9deafd5f00abeeed
I20260808 22:31:39.303509 2359 ts_tablet_manager.cc:1939] T d1aa1be2c714429d9deafd5f00abeeed P dba5657cda984625a81e9fa4dda06b8b: Deleting consensus metadata
I20260808 22:31:39.303681 1359 catalog_manager.cc:4841] TS 9f36d03373d54b269ac978d372329799 (127.31.87.129:40357): tablet d1aa1be2c714429d9deafd5f00abeeed (table test-table [id=6145aa535915492ba6da1a9576344bf0]) successfully deleted
I20260808 22:31:39.304190 1360 catalog_manager.cc:4841] TS dba5657cda984625a81e9fa4dda06b8b (127.31.87.130:44213): tablet d1aa1be2c714429d9deafd5f00abeeed (table test-table [id=6145aa535915492ba6da1a9576344bf0]) successfully deleted
Aug 08 22:31:39 dist-test-slave-266d krb5kdc[1318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228298, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.87.129@KRBTEST.COM
I20260808 22:31:39.314441 2363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:39.304153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48950 (local address 127.31.87.129:40357)
0808 22:31:39.304398 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:39.304402 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:39.304706 (+ 304us) server_negotiation.cc:407] Connection header received
0808 22:31:39.304806 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:39.304809 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:39.304859 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:39.304966 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:39.305642 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:39.306437 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:39.307249 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:39.307566 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:39.307673 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:39.310746 (+ 3073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:39.310759 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:39.310773 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:39.310800 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:39.312754 (+ 1954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:39.313165 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:39.313172 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:39.313174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:39.313272 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:39.313518 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:39.313522 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:39.313523 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:39.313838 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:39.314041 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:39.314148 (+ 107us) server_negotiation.cc:299] Negotiation successful
0808 22:31:39.314272 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":57,"threads_started":1}
W20260808 22:31:39.314930 1699 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48950
I20260808 22:31:39.323400 2368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:39.316516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58036 (local address 127.31.87.190:39551)
0808 22:31:39.316738 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:39.316742 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:39.316755 (+ 13us) server_negotiation.cc:407] Connection header received
0808 22:31:39.316858 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:39.316860 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:39.316904 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:39.316975 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:39.317598 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:39.318120 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:39.318901 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:39.319132 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:39.319218 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:39.320424 (+ 1206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:39.320428 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:39.320431 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:39.320456 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:39.321927 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:39.322285 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:39.322289 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:39.322291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:39.322366 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:39.322596 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:39.322600 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:39.322602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:39.322779 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:39.322894 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:39.323133 (+ 239us) server_negotiation.cc:299] Negotiation successful
0808 22:31:39.323244 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":74,"threads_started":1}
W20260808 22:31:39.323733 1374 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58036
I20260808 22:31:39.324875 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 1524
I20260808 22:31:39.338209 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 1756
I20260808 22:31:39.349166 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 1995
I20260808 22:31:39.366894 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 1334
2026-08-08T22:31:39Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3228 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:31:39 dist-test-slave-266d krb5kdc[2372](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:31:39 dist-test-slave-266d krb5kdc[2372](info): set up 2 sockets
Aug 08 22:31:39 dist-test-slave-266d krb5kdc[2372](info): commencing operation
krb5kdc: starting...
W20260808 22:31:40.531538 32094 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Aug 08 22:31:40 dist-test-slave-266d krb5kdc[2372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:31:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:31:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.87.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
I20260808 22:31:40.644513 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:45895
--webserver_interface=127.31.87.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:44001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:45895
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:31:40.750756 2390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:40.750988 2390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:40.751042 2390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:40.754913 2390 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:31:40.754961 2390 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 22:31:40.754985 2390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:40.755009 2390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:40.755038 2390 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:31:40.755061 2390 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:31:40.759979 2390 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:44001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:45895
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:45895
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.87.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2390
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:40.760869 2390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:40.761801 2390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:40.769608 2400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:40.769626 2395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:40.769685 2390 server_base.cc:1034] running on GCE node
W20260808 22:31:40.769589 2396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:40.770363 2390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:40.771360 2390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:40.772516 2390 hybrid_clock.cc:648] HybridClock initialized: now 1786228300772497 us; error 30 us; skew 500 ppm
Aug 08 22:31:40 dist-test-slave-266d krb5kdc[2372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228300, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:31:40.774978 2390 init.cc:377] Logged in from keytab as kudu/127.31.87.190@KRBTEST.COM (short username kudu)
I20260808 22:31:40.775758 2390 webserver.cc:460] Webserver started at http://127.31.87.190:37343/ using document root <none> and password file <none>
I20260808 22:31:40.776266 2390 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:40.776342 2390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:40.776541 2390 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:40.778505 2390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/master-0/data/instance:
uuid: "256a7c3ccaf0495a895c0b2eab960713"
format_stamp: "Formatted at 2026-08-08 22:31:40 on dist-test-slave-266d"
server_key: "2b58c03d8ad484a04c73531da6726334"
server_key_iv: "6e798ec21eaa8adb1fd99027fb5fa735"
server_key_version: "encryptionkey@0"
I20260808 22:31:40.779052 2390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/master-0/wal/instance:
uuid: "256a7c3ccaf0495a895c0b2eab960713"
format_stamp: "Formatted at 2026-08-08 22:31:40 on dist-test-slave-266d"
server_key: "2b58c03d8ad484a04c73531da6726334"
server_key_iv: "6e798ec21eaa8adb1fd99027fb5fa735"
server_key_version: "encryptionkey@0"
I20260808 22:31:40.782509 2390 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:31:40.784741 2408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:40.785900 2390 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:31:40.786008 2390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "256a7c3ccaf0495a895c0b2eab960713"
format_stamp: "Formatted at 2026-08-08 22:31:40 on dist-test-slave-266d"
server_key: "2b58c03d8ad484a04c73531da6726334"
server_key_iv: "6e798ec21eaa8adb1fd99027fb5fa735"
server_key_version: "encryptionkey@0"
I20260808 22:31:40.786096 2390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:40.798086 2390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:40.800480 2390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:40.800642 2390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:40.807638 2390 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.190:45895
I20260808 22:31:40.807690 2502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.190:45895 every 8 connection(s)
I20260808 22:31:40.808476 2390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:31:40.809350 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 2390
I20260808 22:31:40.809445 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/master-0/wal/instance
I20260808 22:31:40.809675 32094 external_mini_cluster.cc:1428] Setting key 0172ea17a0feae8a665979378c58491e
I20260808 22:31:40.811378 2503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:40.817850 2503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 256a7c3ccaf0495a895c0b2eab960713: Bootstrap starting.
Aug 08 22:31:40 dist-test-slave-266d krb5kdc[2372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:31:40.820401 2503 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 256a7c3ccaf0495a895c0b2eab960713: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:40.821854 2503 log.cc:824] T 00000000000000000000000000000000 P 256a7c3ccaf0495a895c0b2eab960713: Log is configured to *not* fsync() on all Append() calls
I20260808 22:31:40.823909 2510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:40.810945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60646 (local address 127.31.87.190:45895)
0808 22:31:40.811342 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:40.811360 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:40.811402 (+ 42us) server_negotiation.cc:407] Connection header received
0808 22:31:40.812199 (+ 797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:40.812222 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:40.812604 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:40.812950 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:40.813584 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:40.815025 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:40.815826 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:40.816505 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:40.816660 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:40.819095 (+ 2435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:40.819110 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:40.819126 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:40.819156 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:40.820974 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:40.821400 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:40.821406 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:40.821412 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:40.821542 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:40.821768 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:40.821771 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:40.821773 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:40.822271 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:40.822416 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:40.822723 (+ 307us) server_negotiation.cc:299] Negotiation successful
0808 22:31:40.822992 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":109,"threads_started":1}
I20260808 22:31:40.824028 2503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 256a7c3ccaf0495a895c0b2eab960713: No bootstrap required, opened a new log
I20260808 22:31:40.827703 2503 raft_consensus.cc:348] T 00000000000000000000000000000000 P 256a7c3ccaf0495a895c0b2eab960713 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "256a7c3ccaf0495a895c0b2eab960713" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45895 } }
I20260808 22:31:40.827906 2503 raft_consensus.cc:374] T 00000000000000000000000000000000 P 256a7c3ccaf0495a895c0b2eab960713 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:40.827955 2503 raft_consensus.cc:729] T 00000000000000000000000000000000 P 256a7c3ccaf0495a895c0b2eab960713 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 256a7c3ccaf0495a895c0b2eab960713, State: Initialized, Role: FOLLOWER
I20260808 22:31:40.828344 2503 consensus_queue.cc:260] T 00000000000000000000000000000000 P 256a7c3ccaf0495a895c0b2eab960713 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "256a7c3ccaf0495a895c0b2eab960713" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45895 } }
I20260808 22:31:40.828464 2503 raft_consensus.cc:388] T 00000000000000000000000000000000 P 256a7c3ccaf0495a895c0b2eab960713 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:31:40.828522 2503 raft_consensus.cc:482] T 00000000000000000000000000000000 P 256a7c3ccaf0495a895c0b2eab960713 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:31:40.828619 2503 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 256a7c3ccaf0495a895c0b2eab960713 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:40.829481 2503 raft_consensus.cc:504] T 00000000000000000000000000000000 P 256a7c3ccaf0495a895c0b2eab960713 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "256a7c3ccaf0495a895c0b2eab960713" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45895 } }
I20260808 22:31:40.829797 2503 leader_election.cc:302] T 00000000000000000000000000000000 P 256a7c3ccaf0495a895c0b2eab960713 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 256a7c3ccaf0495a895c0b2eab960713; no voters:
I20260808 22:31:40.830056 2503 leader_election.cc:288] T 00000000000000000000000000000000 P 256a7c3ccaf0495a895c0b2eab960713 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:31:40.830159 2512 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 256a7c3ccaf0495a895c0b2eab960713 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:31:40.830389 2512 raft_consensus.cc:686] T 00000000000000000000000000000000 P 256a7c3ccaf0495a895c0b2eab960713 [term 1 LEADER]: Becoming Leader. State: Replica: 256a7c3ccaf0495a895c0b2eab960713, State: Running, Role: LEADER
I20260808 22:31:40.830742 2512 consensus_queue.cc:237] T 00000000000000000000000000000000 P 256a7c3ccaf0495a895c0b2eab960713 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "256a7c3ccaf0495a895c0b2eab960713" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45895 } }
I20260808 22:31:40.831192 2503 sys_catalog.cc:564] T 00000000000000000000000000000000 P 256a7c3ccaf0495a895c0b2eab960713 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:31:40.832484 2514 sys_catalog.cc:455] T 00000000000000000000000000000000 P 256a7c3ccaf0495a895c0b2eab960713 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 256a7c3ccaf0495a895c0b2eab960713. Latest consensus state: current_term: 1 leader_uuid: "256a7c3ccaf0495a895c0b2eab960713" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "256a7c3ccaf0495a895c0b2eab960713" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45895 } } }
I20260808 22:31:40.832616 2514 sys_catalog.cc:458] T 00000000000000000000000000000000 P 256a7c3ccaf0495a895c0b2eab960713 [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:40.832595 2513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 256a7c3ccaf0495a895c0b2eab960713 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "256a7c3ccaf0495a895c0b2eab960713" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "256a7c3ccaf0495a895c0b2eab960713" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45895 } } }
I20260808 22:31:40.832721 2513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 256a7c3ccaf0495a895c0b2eab960713 [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:40.833630 2528 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:31:40.836100 2528 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:31:40.841178 2528 catalog_manager.cc:1347] Generated new cluster ID: 48689c3e54ad4d71922a91416aa3ffbc
I20260808 22:31:40.841233 2528 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:31:40.855029 2528 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:31:40.856164 2528 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:31:40.863598 2528 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 256a7c3ccaf0495a895c0b2eab960713: Generated new TSK 0
I20260808 22:31:40.864321 2528 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.87.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab.
Entry for principal kudu/127.31.87.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab.
I20260808 22:31:40.891566 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.129:0
--local_ip_for_outbound_sockets=127.31.87.129
--webserver_interface=127.31.87.129
--webserver_port=0
--tserver_master_addrs=127.31.87.190:45895
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab
--principal=kudu/127.31.87.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:44001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:31:40.999917 2550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:41.000144 2550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:41.000193 2550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:41.004129 2550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:41.004182 2550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:41.004226 2550 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 22:31:41.004254 2550 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 22:31:41.004310 2550 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.129
I20260808 22:31:41.009609 2550 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:44001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab
--principal=kudu/127.31.87.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.87.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.87.190:45895
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2550
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.129
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:41.010588 2550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:41.011543 2550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:41.020578 2563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:41.020712 2550 server_base.cc:1034] running on GCE node
W20260808 22:31:41.020598 2567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:41.020637 2564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:41.021194 2550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:41.021790 2550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:41.022955 2550 hybrid_clock.cc:648] HybridClock initialized: now 1786228301022944 us; error 30 us; skew 500 ppm
Aug 08 22:31:41 dist-test-slave-266d krb5kdc[2372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228301, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:31:41.025851 2550 init.cc:377] Logged in from keytab as kudu/127.31.87.129@KRBTEST.COM (short username kudu)
I20260808 22:31:41.026690 2550 webserver.cc:460] Webserver started at http://127.31.87.129:45377/ using document root <none> and password file <none>
I20260808 22:31:41.027202 2550 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:41.027272 2550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:41.027467 2550 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:41.029455 2550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-0/data/instance:
uuid: "c1071b6848624168ac96b1e5419ae225"
format_stamp: "Formatted at 2026-08-08 22:31:41 on dist-test-slave-266d"
server_key: "627cbcdc1f7e77f4c4646f47cda490e3"
server_key_iv: "41a1a58264769a4e489feab05446fa04"
server_key_version: "encryptionkey@0"
I20260808 22:31:41.029963 2550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-0/wal/instance:
uuid: "c1071b6848624168ac96b1e5419ae225"
format_stamp: "Formatted at 2026-08-08 22:31:41 on dist-test-slave-266d"
server_key: "627cbcdc1f7e77f4c4646f47cda490e3"
server_key_iv: "41a1a58264769a4e489feab05446fa04"
server_key_version: "encryptionkey@0"
I20260808 22:31:41.033497 2550 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:31:41.035651 2577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:41.036850 2550 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:31:41.036957 2550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-0/wal
uuid: "c1071b6848624168ac96b1e5419ae225"
format_stamp: "Formatted at 2026-08-08 22:31:41 on dist-test-slave-266d"
server_key: "627cbcdc1f7e77f4c4646f47cda490e3"
server_key_iv: "41a1a58264769a4e489feab05446fa04"
server_key_version: "encryptionkey@0"
I20260808 22:31:41.037045 2550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:41.064380 2550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:41.066939 2550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:41.067101 2550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:41.068439 2550 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:31:41.068487 2550 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:41.068536 2550 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:31:41.068562 2550 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 22:31:41 dist-test-slave-266d krb5kdc[2372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228301, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.129@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:31:41.078549 2550 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.129:36811
I20260808 22:31:41.078586 2754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.129:36811 every 8 connection(s)
I20260808 22:31:41.079417 2550 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
I20260808 22:31:41.081279 2510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:41.069356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:52221 (local address 127.31.87.190:45895)
0808 22:31:41.069503 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:41.069509 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:41.070336 (+ 827us) server_negotiation.cc:407] Connection header received
0808 22:31:41.071511 (+ 1175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:41.071515 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:41.071574 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:41.071659 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:41.072838 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.073412 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.074556 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.074790 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.074880 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:41.077828 (+ 2948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:41.077832 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:41.077835 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:41.077864 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:41.079388 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:41.079895 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:41.079899 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:41.079901 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:41.080003 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:41.080365 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:41.080367 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:41.080369 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:41.080549 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:41.080629 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:41.081023 (+ 394us) server_negotiation.cc:299] Negotiation successful
0808 22:31:41.081153 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 22:31:41.082049 2596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:41.069711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45895 (local address 127.31.87.129:52221)
0808 22:31:41.070170 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:41.070221 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:41.071252 (+ 1031us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:41.071834 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:41.071841 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:41.072173 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:41.072681 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:41.072702 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.073524 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.073528 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:41.074436 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:41.074446 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.074941 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.074945 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:41.075110 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:41.075622 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:31:41.075641 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:41.077657 (+ 2016us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:41.079499 (+ 1842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:41.079506 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:41.079521 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:41.079776 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:41.080107 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:41.080110 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:41.080112 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:41.080270 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:41.080641 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:41.080645 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:41.080846 (+ 201us) client_negotiation.cc:769] Sending connection context
0808 22:31:41.080987 (+ 141us) client_negotiation.cc:240] Negotiation successful
0808 22:31:41.081248 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":106,"threads_started":1}
I20260808 22:31:41.083230 2756 heartbeater.cc:347] Connected to a master server at 127.31.87.190:45895
I20260808 22:31:41.083365 2756 heartbeater.cc:464] Registering TS with master...
I20260808 22:31:41.083818 2756 heartbeater.cc:511] Master 127.31.87.190:45895 requested a full tablet report, sending...
I20260808 22:31:41.085165 2437 ts_manager.cc:194] Registered new tserver with Master: c1071b6848624168ac96b1e5419ae225 (127.31.87.129:36811)
I20260808 22:31:41.086599 2437 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.129@KRBTEST.COM'} at 127.31.87.129:52221
I20260808 22:31:41.087525 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 2550
I20260808 22:31:41.087663 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-0/wal/instance
I20260808 22:31:41.087908 32094 external_mini_cluster.cc:1428] Setting key 485696f635545ddeee4e456de78ebac9
I20260808 22:31:41.091738 2765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:41.085152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45895 (local address 127.31.87.129:48623)
0808 22:31:41.085418 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:41.085429 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:41.085502 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:41.085689 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:41.085691 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:41.085836 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:41.085924 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:41.085929 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.086846 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.086849 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:41.087476 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:41.087488 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.088049 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.088053 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:41.088155 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:41.088600 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:31:41.088614 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:41.088920 (+ 306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:41.090470 (+ 1550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:41.090473 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:41.090475 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:41.090696 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:41.090918 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:41.090921 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:41.090922 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:41.091006 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:41.091359 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:41.091361 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:41.091432 (+ 71us) client_negotiation.cc:769] Sending connection context
0808 22:31:41.091507 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 22:31:41.091601 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":79,"threads_started":1}
I20260808 22:31:41.091827 2510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:41.085221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:48623 (local address 127.31.87.190:45895)
0808 22:31:41.085354 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:41.085358 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:41.085478 (+ 120us) server_negotiation.cc:407] Connection header received
0808 22:31:41.085557 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:41.085559 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:41.085595 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:41.085663 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:41.086019 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.086717 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.087595 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.087925 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.088025 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:41.089032 (+ 1007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:41.089035 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:41.089038 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:41.089059 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:41.090367 (+ 1308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:41.090785 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:41.090787 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:41.090789 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:41.090851 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:41.091083 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:41.091086 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:41.091087 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:41.091276 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:41.091366 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:41.091566 (+ 200us) server_negotiation.cc:299] Negotiation successful
0808 22:31:41.091675 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
WARNING: no policy specified for kudu/127.31.87.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab.
Entry for principal kudu/127.31.87.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab.
I20260808 22:31:41.113138 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.130:0
--local_ip_for_outbound_sockets=127.31.87.130
--webserver_interface=127.31.87.130
--webserver_port=0
--tserver_master_addrs=127.31.87.190:45895
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab
--principal=kudu/127.31.87.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:44001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:31:41.217998 2770 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:41.218209 2770 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:41.218257 2770 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:41.222097 2770 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:41.222144 2770 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:41.222193 2770 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 22:31:41.222219 2770 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 22:31:41.222270 2770 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.130
I20260808 22:31:41.227162 2770 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:44001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab
--principal=kudu/127.31.87.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.87.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.87.190:45895
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2770
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.130
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:41.228019 2770 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:41.228907 2770 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:41.238023 2781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:41.238049 2780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:41.238065 2784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:41.238322 2770 server_base.cc:1034] running on GCE node
I20260808 22:31:41.238664 2770 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:41.239194 2770 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:41.240350 2770 hybrid_clock.cc:648] HybridClock initialized: now 1786228301240331 us; error 32 us; skew 500 ppm
Aug 08 22:31:41 dist-test-slave-266d krb5kdc[2372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228301, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:31:41.242858 2770 init.cc:377] Logged in from keytab as kudu/127.31.87.130@KRBTEST.COM (short username kudu)
I20260808 22:31:41.243624 2770 webserver.cc:460] Webserver started at http://127.31.87.130:34707/ using document root <none> and password file <none>
I20260808 22:31:41.244118 2770 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:41.244180 2770 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:41.244357 2770 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:41.246208 2770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-1/data/instance:
uuid: "b1cd70f211c04db89dce7456ba0c76a6"
format_stamp: "Formatted at 2026-08-08 22:31:41 on dist-test-slave-266d"
server_key: "013cadfb85a8afa2261f3a8eb52096e8"
server_key_iv: "13c1f1e61290bdffbabfa132ef06bc61"
server_key_version: "encryptionkey@0"
I20260808 22:31:41.246701 2770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-1/wal/instance:
uuid: "b1cd70f211c04db89dce7456ba0c76a6"
format_stamp: "Formatted at 2026-08-08 22:31:41 on dist-test-slave-266d"
server_key: "013cadfb85a8afa2261f3a8eb52096e8"
server_key_iv: "13c1f1e61290bdffbabfa132ef06bc61"
server_key_version: "encryptionkey@0"
I20260808 22:31:41.250059 2770 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:31:41.252228 2796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:41.253351 2770 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:31:41.253449 2770 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-1/wal
uuid: "b1cd70f211c04db89dce7456ba0c76a6"
format_stamp: "Formatted at 2026-08-08 22:31:41 on dist-test-slave-266d"
server_key: "013cadfb85a8afa2261f3a8eb52096e8"
server_key_iv: "13c1f1e61290bdffbabfa132ef06bc61"
server_key_version: "encryptionkey@0"
I20260808 22:31:41.253535 2770 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:41.277884 2770 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:41.280505 2770 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:41.280678 2770 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:41.282156 2770 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:31:41.282205 2770 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:41.282258 2770 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:31:41.282284 2770 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 22:31:41 dist-test-slave-266d krb5kdc[2372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228301, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.130@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:31:41.292371 2770 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.130:41725
I20260808 22:31:41.292618 2987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.130:41725 every 8 connection(s)
I20260808 22:31:41.293255 2770 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
I20260808 22:31:41.295149 2510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:41.282867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:36083 (local address 127.31.87.190:45895)
0808 22:31:41.282992 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:41.282996 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:41.284032 (+ 1036us) server_negotiation.cc:407] Connection header received
0808 22:31:41.285172 (+ 1140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:41.285175 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:41.285236 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:41.285287 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:41.286575 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.287190 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.288418 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.288699 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.288798 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:41.291568 (+ 2770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:41.291571 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:41.291573 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:41.291599 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:41.293017 (+ 1418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:41.293596 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:41.293600 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:41.293602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:41.293702 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:41.294240 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:41.294244 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:41.294245 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:41.294433 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:41.294512 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:41.294934 (+ 422us) server_negotiation.cc:299] Negotiation successful
0808 22:31:41.295033 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 22:31:41.295881 2810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:41.283359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45895 (local address 127.31.87.130:36083)
0808 22:31:41.283898 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:41.283940 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:41.284891 (+ 951us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:41.285633 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:41.285639 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:41.285944 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:41.286420 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:41.286435 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.287314 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.287317 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:41.288307 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:41.288317 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.288848 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.288853 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:41.289067 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:41.289629 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:31:41.289648 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:41.291399 (+ 1751us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:41.293145 (+ 1746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:41.293150 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:41.293163 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:41.293491 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:41.293950 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:41.293953 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:41.293955 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:41.294126 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:41.294511 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:41.294516 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:41.294745 (+ 229us) client_negotiation.cc:769] Sending connection context
0808 22:31:41.294881 (+ 136us) client_negotiation.cc:240] Negotiation successful
0808 22:31:41.295116 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":132,"threads_started":1}
I20260808 22:31:41.296954 2989 heartbeater.cc:347] Connected to a master server at 127.31.87.190:45895
I20260808 22:31:41.297120 2989 heartbeater.cc:464] Registering TS with master...
I20260808 22:31:41.297560 2989 heartbeater.cc:511] Master 127.31.87.190:45895 requested a full tablet report, sending...
I20260808 22:31:41.298480 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 2770
I20260808 22:31:41.298518 2433 ts_manager.cc:194] Registered new tserver with Master: b1cd70f211c04db89dce7456ba0c76a6 (127.31.87.130:41725)
I20260808 22:31:41.298584 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-1/wal/instance
I20260808 22:31:41.298910 32094 external_mini_cluster.cc:1428] Setting key 2b1687d1af8285880c3510a49f0abcc2
I20260808 22:31:41.299276 2433 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.130@KRBTEST.COM'} at 127.31.87.130:36083
I20260808 22:31:41.305981 3003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:41.299027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45895 (local address 127.31.87.130:52399)
0808 22:31:41.299587 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:41.299599 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:41.299690 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:41.299903 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:41.299906 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:41.300239 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:41.300367 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:41.300373 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.301057 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.301061 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:41.301654 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:41.301662 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.302067 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.302070 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:41.302158 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:41.302651 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:31:41.302666 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:41.302956 (+ 290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:41.304569 (+ 1613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:41.304571 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:41.304573 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:41.304769 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:41.305028 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:41.305030 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:41.305031 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:41.305125 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:41.305586 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:41.305588 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:41.305679 (+ 91us) client_negotiation.cc:769] Sending connection context
0808 22:31:41.305759 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 22:31:41.305855 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":88,"threads_started":1}
I20260808 22:31:41.306084 2510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:41.299045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:52399 (local address 127.31.87.190:45895)
0808 22:31:41.299142 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:41.299146 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:41.299671 (+ 525us) server_negotiation.cc:407] Connection header received
0808 22:31:41.299774 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:41.299777 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:41.299817 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:41.299886 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:41.300472 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.300946 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.301758 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.301974 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.302056 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:41.303077 (+ 1021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:41.303080 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:41.303083 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:41.303105 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:41.304464 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:41.304858 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:41.304861 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:41.304864 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:41.304946 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:41.305225 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:41.305228 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:41.305230 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:41.305497 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:41.305597 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:41.305832 (+ 235us) server_negotiation.cc:299] Negotiation successful
0808 22:31:41.305942 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
WARNING: no policy specified for kudu/127.31.87.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab.
Entry for principal kudu/127.31.87.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab.
I20260808 22:31:41.325562 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.131:0
--local_ip_for_outbound_sockets=127.31.87.131
--webserver_interface=127.31.87.131
--webserver_port=0
--tserver_master_addrs=127.31.87.190:45895
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab
--principal=kudu/127.31.87.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:44001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:31:41.434612 3007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:41.434842 3007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:41.434906 3007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:41.438823 3007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:41.438875 3007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:41.438930 3007 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 22:31:41.438956 3007 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 22:31:41.439002 3007 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.131
I20260808 22:31:41.444109 3007 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:44001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab
--principal=kudu/127.31.87.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.87.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.87.190:45895
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.131
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:41.445021 3007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:41.445966 3007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:41.454900 3026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:41.454952 3021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:41.454912 3022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:41.455169 3007 server_base.cc:1034] running on GCE node
I20260808 22:31:41.455606 3007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:41.456197 3007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:41.457360 3007 hybrid_clock.cc:648] HybridClock initialized: now 1786228301457346 us; error 31 us; skew 500 ppm
Aug 08 22:31:41 dist-test-slave-266d krb5kdc[2372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228301, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:31:41.459905 3007 init.cc:377] Logged in from keytab as kudu/127.31.87.131@KRBTEST.COM (short username kudu)
I20260808 22:31:41.460724 3007 webserver.cc:460] Webserver started at http://127.31.87.131:42191/ using document root <none> and password file <none>
I20260808 22:31:41.461285 3007 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:41.461346 3007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:41.461536 3007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:41.463413 3007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-2/data/instance:
uuid: "130f382ce0ca4e189e6cecf96b49ea59"
format_stamp: "Formatted at 2026-08-08 22:31:41 on dist-test-slave-266d"
server_key: "b6c17679e0b01671b3596c438c24a6eb"
server_key_iv: "5be28c56c6ca3d41c9cee66a8e0a2982"
server_key_version: "encryptionkey@0"
I20260808 22:31:41.463922 3007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-2/wal/instance:
uuid: "130f382ce0ca4e189e6cecf96b49ea59"
format_stamp: "Formatted at 2026-08-08 22:31:41 on dist-test-slave-266d"
server_key: "b6c17679e0b01671b3596c438c24a6eb"
server_key_iv: "5be28c56c6ca3d41c9cee66a8e0a2982"
server_key_version: "encryptionkey@0"
I20260808 22:31:41.467522 3007 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 22:31:41.469741 3040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:41.470920 3007 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:31:41.471020 3007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-2/wal
uuid: "130f382ce0ca4e189e6cecf96b49ea59"
format_stamp: "Formatted at 2026-08-08 22:31:41 on dist-test-slave-266d"
server_key: "b6c17679e0b01671b3596c438c24a6eb"
server_key_iv: "5be28c56c6ca3d41c9cee66a8e0a2982"
server_key_version: "encryptionkey@0"
I20260808 22:31:41.471153 3007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:41.482589 3007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:41.485494 3007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:41.485661 3007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:41.487016 3007 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:31:41.487075 3007 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:41.487128 3007 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:31:41.487154 3007 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 22:31:41 dist-test-slave-266d krb5kdc[2372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228301, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.131@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:31:41.497725 3007 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.131:45421
I20260808 22:31:41.497733 3236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.131:45421 every 8 connection(s)
I20260808 22:31:41.498695 3007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
I20260808 22:31:41.500131 2510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:41.487649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:43317 (local address 127.31.87.190:45895)
0808 22:31:41.487799 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:41.487805 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:41.489194 (+ 1389us) server_negotiation.cc:407] Connection header received
0808 22:31:41.490248 (+ 1054us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:41.490251 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:41.490309 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:41.490376 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:41.491568 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.492360 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.493577 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.493805 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.493895 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:41.496484 (+ 2589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:41.496487 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:41.496489 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:41.496513 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:41.498211 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:41.498666 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:41.498670 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:41.498672 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:41.498767 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:41.499124 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:41.499128 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:41.499129 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:41.499338 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:41.499431 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:41.499909 (+ 478us) server_negotiation.cc:299] Negotiation successful
0808 22:31:41.500007 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 22:31:41.500819 3059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:41.488522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45895 (local address 127.31.87.131:43317)
0808 22:31:41.488997 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:41.489046 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:41.490040 (+ 994us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:41.490521 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:41.490527 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:41.490968 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:41.491424 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:41.491437 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.492473 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.492477 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:41.493464 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:41.493474 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.493940 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.493944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:41.494099 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:41.494589 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:31:41.494605 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:41.496323 (+ 1718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:41.498314 (+ 1991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:41.498319 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:41.498329 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:41.498567 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:41.498876 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:41.498880 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:41.498881 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:41.499017 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:41.499485 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:41.499491 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:41.499697 (+ 206us) client_negotiation.cc:769] Sending connection context
0808 22:31:41.499843 (+ 146us) client_negotiation.cc:240] Negotiation successful
0808 22:31:41.500058 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":126,"threads_started":1}
I20260808 22:31:41.500972 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 3007
I20260808 22:31:41.501108 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-2/wal/instance
I20260808 22:31:41.501400 32094 external_mini_cluster.cc:1428] Setting key 9ceb5c53ca9a3c5b99734669a60e8cc1
I20260808 22:31:41.502246 3238 heartbeater.cc:347] Connected to a master server at 127.31.87.190:45895
I20260808 22:31:41.502529 3238 heartbeater.cc:464] Registering TS with master...
I20260808 22:31:41.503095 3238 heartbeater.cc:511] Master 127.31.87.190:45895 requested a full tablet report, sending...
I20260808 22:31:41.504079 2433 ts_manager.cc:194] Registered new tserver with Master: 130f382ce0ca4e189e6cecf96b49ea59 (127.31.87.131:45421)
I20260808 22:31:41.504683 2433 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.131@KRBTEST.COM'} at 127.31.87.131:43317
I20260808 22:31:41.510437 2510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:41.504305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:36763 (local address 127.31.87.190:45895)
0808 22:31:41.504426 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:41.504430 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:41.504701 (+ 271us) server_negotiation.cc:407] Connection header received
0808 22:31:41.504794 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:41.504796 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:41.504836 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:41.504908 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:41.505361 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.505838 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.506597 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.506805 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.506887 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:41.507804 (+ 917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:41.507808 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:41.507810 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:41.507834 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:41.509038 (+ 1204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:41.509423 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:41.509427 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:41.509428 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:41.509496 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:41.509733 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:41.509736 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:41.509738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:41.509888 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:41.509971 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:41.510217 (+ 246us) server_negotiation.cc:299] Negotiation successful
0808 22:31:41.510310 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 22:31:41.510452 3246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:41.504367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45895 (local address 127.31.87.131:36763)
0808 22:31:41.504628 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:41.504640 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:41.504716 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:41.504934 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:41.504937 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:41.505164 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:41.505259 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:41.505265 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.505951 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.505954 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:41.506494 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:41.506502 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.506899 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.506903 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:41.506987 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:41.507395 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:31:41.507409 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:41.507681 (+ 272us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:41.509158 (+ 1477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:41.509162 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:41.509164 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:41.509338 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:41.509576 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:41.509579 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:41.509581 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:41.509656 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:41.509975 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:41.509979 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:41.510063 (+ 84us) client_negotiation.cc:769] Sending connection context
0808 22:31:41.510155 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 22:31:41.510280 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":100,"threads_started":1}
I20260808 22:31:41.514815 32094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 22:31:41 dist-test-slave-266d krb5kdc[2372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228301, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 22:31:41 dist-test-slave-266d krb5kdc[2372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228301, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:31:41.538823 2510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:41.531727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60648 (local address 127.31.87.190:45895)
0808 22:31:41.531824 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:41.531828 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:41.531838 (+ 10us) server_negotiation.cc:407] Connection header received
0808 22:31:41.531870 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:41.531872 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:41.531906 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:41.531960 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:41.532588 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.533113 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.533941 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.534150 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.534228 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:41.536348 (+ 2120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:41.536351 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:41.536353 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:41.536376 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:41.537596 (+ 1220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:41.537970 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:41.537972 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:41.537973 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:41.538044 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:41.538218 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:41.538220 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:41.538221 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:41.538367 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:41.538459 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:41.538597 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 22:31:41.538708 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 22:31:41.541584 2433 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60648:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:31:41.544442 2433 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:31:41.558609 3270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:41.552424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:36811 (local address 127.0.0.1:33816)
0808 22:31:41.553407 (+ 983us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:41.553418 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:41.553489 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:41.554189 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:41.554195 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:41.554232 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:31:41.554463 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:41.554469 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.555901 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.555904 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:41.556993 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:41.557003 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.558064 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.558067 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:41.558178 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:41.558220 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 22:31:41.558343 (+ 123us) client_negotiation.cc:240] Negotiation successful
0808 22:31:41.558453 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":928,"mutex_wait_us":40,"spinlock_wait_cycles":6144,"thread_start_us":92,"threads_started":1}
I20260808 22:31:41.558633 3269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:41.552816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33816 (local address 127.31.87.129:36811)
0808 22:31:41.553696 (+ 880us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:41.553711 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:41.553730 (+ 19us) server_negotiation.cc:407] Connection header received
0808 22:31:41.553792 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:41.553796 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:41.553939 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:41.554051 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:31:41.554636 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.555801 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.557129 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.557940 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.558040 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:41.558203 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:41.558383 (+ 180us) server_negotiation.cc:299] Negotiation successful
0808 22:31:41.558479 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":785,"thread_start_us":74,"threads_started":1}
I20260808 22:31:41.559784 3266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:41.552817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.130:41725 (local address 127.0.0.1:43404)
0808 22:31:41.553836 (+ 1019us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:41.553847 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:41.553937 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:41.554329 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:41.554332 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:41.554349 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:31:41.554472 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:41.554478 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.557130 (+ 2652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.557132 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:41.557957 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:41.557966 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.559409 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.559413 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:41.559494 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:41.559516 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 22:31:41.559579 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 22:31:41.559640 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":971,"mutex_wait_us":5,"thread_start_us":93,"threads_started":1}
I20260808 22:31:41.559916 3267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:41.552288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.131:45421 (local address 127.0.0.1:34584)
0808 22:31:41.553241 (+ 953us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:41.553292 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:41.553422 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:41.554189 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:41.554195 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:41.554234 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:31:41.554462 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:41.554469 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.556962 (+ 2493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.556965 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:41.558131 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:41.558142 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.559277 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.559280 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:41.559364 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:41.559587 (+ 223us) client_negotiation.cc:769] Sending connection context
0808 22:31:41.559670 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 22:31:41.559741 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":766,"mutex_wait_us":80,"spinlock_wait_cycles":6016,"thread_start_us":82,"threads_started":1}
I20260808 22:31:41.559930 3268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:41.552435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34584 (local address 127.31.87.131:45421)
0808 22:31:41.553250 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:41.553275 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:41.553561 (+ 286us) server_negotiation.cc:407] Connection header received
0808 22:31:41.553645 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:41.553651 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:41.553866 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:41.554016 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:31:41.555952 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.556854 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.558270 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.559009 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.559124 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:41.559498 (+ 374us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:41.559716 (+ 218us) server_negotiation.cc:299] Negotiation successful
0808 22:31:41.559804 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":718,"thread_start_us":94,"threads_started":1}
I20260808 22:31:41.560226 3272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:41.553272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43404 (local address 127.31.87.130:41725)
0808 22:31:41.553735 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:41.553744 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:41.553933 (+ 189us) server_negotiation.cc:407] Connection header received
0808 22:31:41.554051 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:41.554056 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:41.554208 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:41.554334 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:31:41.554573 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.555801 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.558091 (+ 2290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.558900 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.558994 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:41.559347 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:41.560006 (+ 659us) server_negotiation.cc:299] Negotiation successful
0808 22:31:41.560094 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":111,"threads_started":1}
I20260808 22:31:41.562047 2651 tablet_service.cc:1467] Processing CreateTablet for tablet 362f2fe32f2b492a967b8a3718f5ef8f (DEFAULT_TABLE table=test-table [id=4301b3d9d08e4432b30dbff02c40ece0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:31:41.562754 2871 tablet_service.cc:1467] Processing CreateTablet for tablet 362f2fe32f2b492a967b8a3718f5ef8f (DEFAULT_TABLE table=test-table [id=4301b3d9d08e4432b30dbff02c40ece0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:31:41.563176 2651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 362f2fe32f2b492a967b8a3718f5ef8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:41.562915 3119 tablet_service.cc:1467] Processing CreateTablet for tablet 362f2fe32f2b492a967b8a3718f5ef8f (DEFAULT_TABLE table=test-table [id=4301b3d9d08e4432b30dbff02c40ece0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:31:41.563717 2871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 362f2fe32f2b492a967b8a3718f5ef8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:41.563817 3119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 362f2fe32f2b492a967b8a3718f5ef8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:41.569734 3275 tablet_bootstrap.cc:492] T 362f2fe32f2b492a967b8a3718f5ef8f P 130f382ce0ca4e189e6cecf96b49ea59: Bootstrap starting.
I20260808 22:31:41.571219 3277 tablet_bootstrap.cc:492] T 362f2fe32f2b492a967b8a3718f5ef8f P c1071b6848624168ac96b1e5419ae225: Bootstrap starting.
I20260808 22:31:41.571954 3275 tablet_bootstrap.cc:654] T 362f2fe32f2b492a967b8a3718f5ef8f P 130f382ce0ca4e189e6cecf96b49ea59: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:41.572733 3275 log.cc:824] T 362f2fe32f2b492a967b8a3718f5ef8f P 130f382ce0ca4e189e6cecf96b49ea59: Log is configured to *not* fsync() on all Append() calls
I20260808 22:31:41.572983 3277 tablet_bootstrap.cc:654] T 362f2fe32f2b492a967b8a3718f5ef8f P c1071b6848624168ac96b1e5419ae225: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:41.573392 3276 tablet_bootstrap.cc:492] T 362f2fe32f2b492a967b8a3718f5ef8f P b1cd70f211c04db89dce7456ba0c76a6: Bootstrap starting.
I20260808 22:31:41.573724 3277 log.cc:824] T 362f2fe32f2b492a967b8a3718f5ef8f P c1071b6848624168ac96b1e5419ae225: Log is configured to *not* fsync() on all Append() calls
I20260808 22:31:41.574868 3275 tablet_bootstrap.cc:492] T 362f2fe32f2b492a967b8a3718f5ef8f P 130f382ce0ca4e189e6cecf96b49ea59: No bootstrap required, opened a new log
I20260808 22:31:41.575057 3275 ts_tablet_manager.cc:1397] T 362f2fe32f2b492a967b8a3718f5ef8f P 130f382ce0ca4e189e6cecf96b49ea59: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 22:31:41.575217 3276 tablet_bootstrap.cc:654] T 362f2fe32f2b492a967b8a3718f5ef8f P b1cd70f211c04db89dce7456ba0c76a6: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:41.575760 3277 tablet_bootstrap.cc:492] T 362f2fe32f2b492a967b8a3718f5ef8f P c1071b6848624168ac96b1e5419ae225: No bootstrap required, opened a new log
I20260808 22:31:41.575937 3277 ts_tablet_manager.cc:1397] T 362f2fe32f2b492a967b8a3718f5ef8f P c1071b6848624168ac96b1e5419ae225: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260808 22:31:41.575927 3276 log.cc:824] T 362f2fe32f2b492a967b8a3718f5ef8f P b1cd70f211c04db89dce7456ba0c76a6: Log is configured to *not* fsync() on all Append() calls
I20260808 22:31:41.577594 3276 tablet_bootstrap.cc:492] T 362f2fe32f2b492a967b8a3718f5ef8f P b1cd70f211c04db89dce7456ba0c76a6: No bootstrap required, opened a new log
I20260808 22:31:41.577802 3276 ts_tablet_manager.cc:1397] T 362f2fe32f2b492a967b8a3718f5ef8f P b1cd70f211c04db89dce7456ba0c76a6: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260808 22:31:41.578886 3275 raft_consensus.cc:348] T 362f2fe32f2b492a967b8a3718f5ef8f P 130f382ce0ca4e189e6cecf96b49ea59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } } peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } }
I20260808 22:31:41.579141 3275 raft_consensus.cc:374] T 362f2fe32f2b492a967b8a3718f5ef8f P 130f382ce0ca4e189e6cecf96b49ea59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:41.579197 3275 raft_consensus.cc:729] T 362f2fe32f2b492a967b8a3718f5ef8f P 130f382ce0ca4e189e6cecf96b49ea59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130f382ce0ca4e189e6cecf96b49ea59, State: Initialized, Role: FOLLOWER
I20260808 22:31:41.579617 3275 consensus_queue.cc:260] T 362f2fe32f2b492a967b8a3718f5ef8f P 130f382ce0ca4e189e6cecf96b49ea59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } } peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } }
I20260808 22:31:41.579573 3277 raft_consensus.cc:348] T 362f2fe32f2b492a967b8a3718f5ef8f P c1071b6848624168ac96b1e5419ae225 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } } peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } }
I20260808 22:31:41.579787 3277 raft_consensus.cc:374] T 362f2fe32f2b492a967b8a3718f5ef8f P c1071b6848624168ac96b1e5419ae225 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:41.579842 3277 raft_consensus.cc:729] T 362f2fe32f2b492a967b8a3718f5ef8f P c1071b6848624168ac96b1e5419ae225 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1071b6848624168ac96b1e5419ae225, State: Initialized, Role: FOLLOWER
I20260808 22:31:41.580277 3277 consensus_queue.cc:260] T 362f2fe32f2b492a967b8a3718f5ef8f P c1071b6848624168ac96b1e5419ae225 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } } peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } }
I20260808 22:31:41.580600 3275 ts_tablet_manager.cc:1428] T 362f2fe32f2b492a967b8a3718f5ef8f P 130f382ce0ca4e189e6cecf96b49ea59: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 22:31:41.580727 3276 raft_consensus.cc:348] T 362f2fe32f2b492a967b8a3718f5ef8f P b1cd70f211c04db89dce7456ba0c76a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } } peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } }
I20260808 22:31:41.580938 3238 heartbeater.cc:503] Master 127.31.87.190:45895 was elected leader, sending a full tablet report...
I20260808 22:31:41.580969 3276 raft_consensus.cc:374] T 362f2fe32f2b492a967b8a3718f5ef8f P b1cd70f211c04db89dce7456ba0c76a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:41.581032 3276 raft_consensus.cc:729] T 362f2fe32f2b492a967b8a3718f5ef8f P b1cd70f211c04db89dce7456ba0c76a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1cd70f211c04db89dce7456ba0c76a6, State: Initialized, Role: FOLLOWER
I20260808 22:31:41.581229 2756 heartbeater.cc:503] Master 127.31.87.190:45895 was elected leader, sending a full tablet report...
I20260808 22:31:41.581411 3276 consensus_queue.cc:260] T 362f2fe32f2b492a967b8a3718f5ef8f P b1cd70f211c04db89dce7456ba0c76a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } } peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } }
I20260808 22:31:41.581960 2989 heartbeater.cc:503] Master 127.31.87.190:45895 was elected leader, sending a full tablet report...
I20260808 22:31:41.582381 3277 ts_tablet_manager.cc:1428] T 362f2fe32f2b492a967b8a3718f5ef8f P c1071b6848624168ac96b1e5419ae225: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260808 22:31:41.583289 3276 ts_tablet_manager.cc:1428] T 362f2fe32f2b492a967b8a3718f5ef8f P b1cd70f211c04db89dce7456ba0c76a6: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 22:31:41.741256 3284 raft_consensus.cc:482] T 362f2fe32f2b492a967b8a3718f5ef8f P 130f382ce0ca4e189e6cecf96b49ea59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:31:41.741460 3284 raft_consensus.cc:504] T 362f2fe32f2b492a967b8a3718f5ef8f P 130f382ce0ca4e189e6cecf96b49ea59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } } peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } }
I20260808 22:31:41.742611 3284 leader_election.cc:288] T 362f2fe32f2b492a967b8a3718f5ef8f P 130f382ce0ca4e189e6cecf96b49ea59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1071b6848624168ac96b1e5419ae225 (127.31.87.129:36811), b1cd70f211c04db89dce7456ba0c76a6 (127.31.87.130:41725)
I20260808 22:31:41.746536 3269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:41.742901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:33567 (local address 127.31.87.129:36811)
0808 22:31:41.743020 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:41.743025 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:41.743165 (+ 140us) server_negotiation.cc:407] Connection header received
0808 22:31:41.743231 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:41.743234 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:41.743285 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:41.743352 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:31:41.743632 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.744334 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.745486 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.746040 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.746138 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:41.746180 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:41.746381 (+ 201us) server_negotiation.cc:299] Negotiation successful
0808 22:31:41.746434 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 22:31:41.746549 3059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:41.742838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:36811 (local address 127.31.87.131:33567)
0808 22:31:41.742999 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:41.743015 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:41.743115 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:41.743372 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:41.743375 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:41.743394 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:31:41.743516 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:41.743523 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.744438 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.744441 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:41.745372 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:41.745381 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.746144 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.746147 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:41.746262 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:41.746288 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 22:31:41.746357 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 22:31:41.746423 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260808 22:31:41.747046 3288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:41.742846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.130:41725 (local address 127.31.87.131:44693)
0808 22:31:41.743166 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:41.743179 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:41.743279 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:41.743512 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:41.743514 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:41.743529 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:31:41.744043 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:41.744052 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.744902 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.744904 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:41.745831 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:41.745842 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.746645 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.746649 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:41.746741 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:41.746766 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 22:31:41.746844 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 22:31:41.746907 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"mutex_wait_us":13,"thread_start_us":130,"threads_started":1}
I20260808 22:31:41.747040 3272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:41.743058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:44693 (local address 127.31.87.130:41725)
0808 22:31:41.743191 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:41.743195 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:41.743250 (+ 55us) server_negotiation.cc:407] Connection header received
0808 22:31:41.743372 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:41.743375 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:41.743420 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:41.743496 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:31:41.744154 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.744794 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.745952 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.746501 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.746624 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:41.746682 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:41.746872 (+ 190us) server_negotiation.cc:299] Negotiation successful
0808 22:31:41.746935 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 22:31:41.747318 2683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "362f2fe32f2b492a967b8a3718f5ef8f" candidate_uuid: "130f382ce0ca4e189e6cecf96b49ea59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1071b6848624168ac96b1e5419ae225" is_pre_election: true
I20260808 22:31:41.747604 2683 raft_consensus.cc:2454] T 362f2fe32f2b492a967b8a3718f5ef8f P c1071b6848624168ac96b1e5419ae225 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130f382ce0ca4e189e6cecf96b49ea59 in term 0.
I20260808 22:31:41.747659 2910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "362f2fe32f2b492a967b8a3718f5ef8f" candidate_uuid: "130f382ce0ca4e189e6cecf96b49ea59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1cd70f211c04db89dce7456ba0c76a6" is_pre_election: true
I20260808 22:31:41.747942 2910 raft_consensus.cc:2454] T 362f2fe32f2b492a967b8a3718f5ef8f P b1cd70f211c04db89dce7456ba0c76a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130f382ce0ca4e189e6cecf96b49ea59 in term 0.
I20260808 22:31:41.748081 3044 leader_election.cc:302] T 362f2fe32f2b492a967b8a3718f5ef8f P 130f382ce0ca4e189e6cecf96b49ea59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130f382ce0ca4e189e6cecf96b49ea59, c1071b6848624168ac96b1e5419ae225; no voters:
I20260808 22:31:41.748347 3284 raft_consensus.cc:2781] T 362f2fe32f2b492a967b8a3718f5ef8f P 130f382ce0ca4e189e6cecf96b49ea59 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:31:41.748435 3284 raft_consensus.cc:482] T 362f2fe32f2b492a967b8a3718f5ef8f P 130f382ce0ca4e189e6cecf96b49ea59 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:31:41.748486 3284 raft_consensus.cc:3037] T 362f2fe32f2b492a967b8a3718f5ef8f P 130f382ce0ca4e189e6cecf96b49ea59 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:41.749521 3284 raft_consensus.cc:504] T 362f2fe32f2b492a967b8a3718f5ef8f P 130f382ce0ca4e189e6cecf96b49ea59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } } peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } }
W20260808 22:31:41.749706 3239 tablet.cc:2369] T 362f2fe32f2b492a967b8a3718f5ef8f P 130f382ce0ca4e189e6cecf96b49ea59: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:31:41.749876 3284 leader_election.cc:288] T 362f2fe32f2b492a967b8a3718f5ef8f P 130f382ce0ca4e189e6cecf96b49ea59 [CANDIDATE]: Term 1 election: Requested vote from peers c1071b6848624168ac96b1e5419ae225 (127.31.87.129:36811), b1cd70f211c04db89dce7456ba0c76a6 (127.31.87.130:41725)
I20260808 22:31:41.750188 2910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "362f2fe32f2b492a967b8a3718f5ef8f" candidate_uuid: "130f382ce0ca4e189e6cecf96b49ea59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1cd70f211c04db89dce7456ba0c76a6"
I20260808 22:31:41.750300 2910 raft_consensus.cc:3037] T 362f2fe32f2b492a967b8a3718f5ef8f P b1cd70f211c04db89dce7456ba0c76a6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:41.751626 2910 raft_consensus.cc:2454] T 362f2fe32f2b492a967b8a3718f5ef8f P b1cd70f211c04db89dce7456ba0c76a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130f382ce0ca4e189e6cecf96b49ea59 in term 1.
I20260808 22:31:41.751861 2683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "362f2fe32f2b492a967b8a3718f5ef8f" candidate_uuid: "130f382ce0ca4e189e6cecf96b49ea59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1071b6848624168ac96b1e5419ae225"
I20260808 22:31:41.751909 3047 leader_election.cc:302] T 362f2fe32f2b492a967b8a3718f5ef8f P 130f382ce0ca4e189e6cecf96b49ea59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130f382ce0ca4e189e6cecf96b49ea59, b1cd70f211c04db89dce7456ba0c76a6; no voters:
I20260808 22:31:41.751978 2683 raft_consensus.cc:3037] T 362f2fe32f2b492a967b8a3718f5ef8f P c1071b6848624168ac96b1e5419ae225 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:41.752049 3284 raft_consensus.cc:2781] T 362f2fe32f2b492a967b8a3718f5ef8f P 130f382ce0ca4e189e6cecf96b49ea59 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:31:41.752249 3284 raft_consensus.cc:686] T 362f2fe32f2b492a967b8a3718f5ef8f P 130f382ce0ca4e189e6cecf96b49ea59 [term 1 LEADER]: Becoming Leader. State: Replica: 130f382ce0ca4e189e6cecf96b49ea59, State: Running, Role: LEADER
I20260808 22:31:41.752552 3284 consensus_queue.cc:237] T 362f2fe32f2b492a967b8a3718f5ef8f P 130f382ce0ca4e189e6cecf96b49ea59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } } peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } }
I20260808 22:31:41.753058 2683 raft_consensus.cc:2454] T 362f2fe32f2b492a967b8a3718f5ef8f P c1071b6848624168ac96b1e5419ae225 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130f382ce0ca4e189e6cecf96b49ea59 in term 1.
I20260808 22:31:41.755641 2433 catalog_manager.cc:5495] T 362f2fe32f2b492a967b8a3718f5ef8f P 130f382ce0ca4e189e6cecf96b49ea59 reported cstate change: term changed from 0 to 1, leader changed from <none> to 130f382ce0ca4e189e6cecf96b49ea59 (127.31.87.131). New cstate: current_term: 1 leader_uuid: "130f382ce0ca4e189e6cecf96b49ea59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } health_report { overall_health: UNKNOWN } } }
W20260808 22:31:41.794342 2991 tablet.cc:2369] T 362f2fe32f2b492a967b8a3718f5ef8f P b1cd70f211c04db89dce7456ba0c76a6: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 22:31:41 dist-test-slave-266d krb5kdc[2372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228300, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:31:41.814328 3300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:41.806686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45895 (local address 127.0.0.1:60650)
0808 22:31:41.806945 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:41.806958 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:41.807055 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:41.807263 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:41.807266 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:41.807678 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:41.807787 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:41.807794 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.808506 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.808510 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:41.809048 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:41.809056 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.809506 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.809509 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:41.809583 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:41.810113 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:31:41.810125 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:41.811435 (+ 1310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:41.812883 (+ 1448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:41.812892 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:41.812898 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:41.813143 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:41.813386 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:41.813389 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:41.813390 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:41.813465 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:41.813799 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:41.813805 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:41.813907 (+ 102us) client_negotiation.cc:769] Sending connection context
0808 22:31:41.813999 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 22:31:41.814144 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":99,"threads_started":1}
I20260808 22:31:41.814327 2510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:41.806721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60650 (local address 127.31.87.190:45895)
0808 22:31:41.806846 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:41.806850 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:41.807022 (+ 172us) server_negotiation.cc:407] Connection header received
0808 22:31:41.807131 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:41.807133 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:41.807174 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:41.807240 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:41.807894 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.808412 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.809192 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.809411 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.809496 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:41.811553 (+ 2057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:41.811557 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:41.811559 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:41.811581 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:41.812792 (+ 1211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:41.813223 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:41.813227 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:41.813229 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:41.813308 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:41.813538 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:41.813541 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:41.813542 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:41.813700 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:41.813782 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:41.814055 (+ 273us) server_negotiation.cc:299] Negotiation successful
0808 22:31:41.814198 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":43}
I20260808 22:31:41.818161 2433 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:60650:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 22:31:41.818912 2433 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:31:41.823367 2651 tablet_service.cc:1467] Processing CreateTablet for tablet 44f6f034569b473f91f98b770b90de3b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7e11d58e2595445cbd6a92c802f042b3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 22:31:41.823433 2871 tablet_service.cc:1467] Processing CreateTablet for tablet 44f6f034569b473f91f98b770b90de3b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7e11d58e2595445cbd6a92c802f042b3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 22:31:41.823621 3119 tablet_service.cc:1467] Processing CreateTablet for tablet 44f6f034569b473f91f98b770b90de3b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7e11d58e2595445cbd6a92c802f042b3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 22:31:41.824038 2651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44f6f034569b473f91f98b770b90de3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:41.824107 2871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44f6f034569b473f91f98b770b90de3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:41.824127 3119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44f6f034569b473f91f98b770b90de3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:41.826545 3277 tablet_bootstrap.cc:492] T 44f6f034569b473f91f98b770b90de3b P c1071b6848624168ac96b1e5419ae225: Bootstrap starting.
I20260808 22:31:41.826931 3275 tablet_bootstrap.cc:492] T 44f6f034569b473f91f98b770b90de3b P 130f382ce0ca4e189e6cecf96b49ea59: Bootstrap starting.
I20260808 22:31:41.827528 3277 tablet_bootstrap.cc:654] T 44f6f034569b473f91f98b770b90de3b P c1071b6848624168ac96b1e5419ae225: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:41.827901 3275 tablet_bootstrap.cc:654] T 44f6f034569b473f91f98b770b90de3b P 130f382ce0ca4e189e6cecf96b49ea59: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:41.828509 3276 tablet_bootstrap.cc:492] T 44f6f034569b473f91f98b770b90de3b P b1cd70f211c04db89dce7456ba0c76a6: Bootstrap starting.
I20260808 22:31:41.829449 3276 tablet_bootstrap.cc:654] T 44f6f034569b473f91f98b770b90de3b P b1cd70f211c04db89dce7456ba0c76a6: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:41.829562 3277 tablet_bootstrap.cc:492] T 44f6f034569b473f91f98b770b90de3b P c1071b6848624168ac96b1e5419ae225: No bootstrap required, opened a new log
I20260808 22:31:41.829648 3277 ts_tablet_manager.cc:1397] T 44f6f034569b473f91f98b770b90de3b P c1071b6848624168ac96b1e5419ae225: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260808 22:31:41.830145 3277 raft_consensus.cc:348] T 44f6f034569b473f91f98b770b90de3b P c1071b6848624168ac96b1e5419ae225 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } } peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } }
I20260808 22:31:41.830264 3277 raft_consensus.cc:374] T 44f6f034569b473f91f98b770b90de3b P c1071b6848624168ac96b1e5419ae225 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:41.830296 3277 raft_consensus.cc:729] T 44f6f034569b473f91f98b770b90de3b P c1071b6848624168ac96b1e5419ae225 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1071b6848624168ac96b1e5419ae225, State: Initialized, Role: FOLLOWER
I20260808 22:31:41.830492 3275 tablet_bootstrap.cc:492] T 44f6f034569b473f91f98b770b90de3b P 130f382ce0ca4e189e6cecf96b49ea59: No bootstrap required, opened a new log
I20260808 22:31:41.830417 3277 consensus_queue.cc:260] T 44f6f034569b473f91f98b770b90de3b P c1071b6848624168ac96b1e5419ae225 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } } peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } }
I20260808 22:31:41.830570 3275 ts_tablet_manager.cc:1397] T 44f6f034569b473f91f98b770b90de3b P 130f382ce0ca4e189e6cecf96b49ea59: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
W20260808 22:31:41.830626 2757 tablet.cc:2369] T 362f2fe32f2b492a967b8a3718f5ef8f P c1071b6848624168ac96b1e5419ae225: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:31:41.830684 3277 ts_tablet_manager.cc:1428] T 44f6f034569b473f91f98b770b90de3b P c1071b6848624168ac96b1e5419ae225: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 22:31:41.830955 3285 tablet_replica.cc:439] T 44f6f034569b473f91f98b770b90de3b P c1071b6848624168ac96b1e5419ae225: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } } peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } } }
I20260808 22:31:41.831045 3275 raft_consensus.cc:348] T 44f6f034569b473f91f98b770b90de3b P 130f382ce0ca4e189e6cecf96b49ea59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } } peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } }
I20260808 22:31:41.831112 3285 tablet_replica.cc:442] T 44f6f034569b473f91f98b770b90de3b P c1071b6848624168ac96b1e5419ae225: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:31:41.831146 3275 raft_consensus.cc:374] T 44f6f034569b473f91f98b770b90de3b P 130f382ce0ca4e189e6cecf96b49ea59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:41.831175 3275 raft_consensus.cc:729] T 44f6f034569b473f91f98b770b90de3b P 130f382ce0ca4e189e6cecf96b49ea59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130f382ce0ca4e189e6cecf96b49ea59, State: Initialized, Role: FOLLOWER
I20260808 22:31:41.831296 3275 consensus_queue.cc:260] T 44f6f034569b473f91f98b770b90de3b P 130f382ce0ca4e189e6cecf96b49ea59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } } peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } }
I20260808 22:31:41.831487 3275 ts_tablet_manager.cc:1428] T 44f6f034569b473f91f98b770b90de3b P 130f382ce0ca4e189e6cecf96b49ea59: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 22:31:41.831521 3284 tablet_replica.cc:439] T 44f6f034569b473f91f98b770b90de3b P 130f382ce0ca4e189e6cecf96b49ea59: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } } peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } } }
I20260808 22:31:41.831651 3284 tablet_replica.cc:442] T 44f6f034569b473f91f98b770b90de3b P 130f382ce0ca4e189e6cecf96b49ea59: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:31:41.832360 3276 tablet_bootstrap.cc:492] T 44f6f034569b473f91f98b770b90de3b P b1cd70f211c04db89dce7456ba0c76a6: No bootstrap required, opened a new log
I20260808 22:31:41.832438 3276 ts_tablet_manager.cc:1397] T 44f6f034569b473f91f98b770b90de3b P b1cd70f211c04db89dce7456ba0c76a6: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 22:31:41.832875 3276 raft_consensus.cc:348] T 44f6f034569b473f91f98b770b90de3b P b1cd70f211c04db89dce7456ba0c76a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } } peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } }
I20260808 22:31:41.832988 3276 raft_consensus.cc:374] T 44f6f034569b473f91f98b770b90de3b P b1cd70f211c04db89dce7456ba0c76a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:41.833024 3276 raft_consensus.cc:729] T 44f6f034569b473f91f98b770b90de3b P b1cd70f211c04db89dce7456ba0c76a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1cd70f211c04db89dce7456ba0c76a6, State: Initialized, Role: FOLLOWER
I20260808 22:31:41.833181 3276 consensus_queue.cc:260] T 44f6f034569b473f91f98b770b90de3b P b1cd70f211c04db89dce7456ba0c76a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } } peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } }
I20260808 22:31:41.833436 3276 ts_tablet_manager.cc:1428] T 44f6f034569b473f91f98b770b90de3b P b1cd70f211c04db89dce7456ba0c76a6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 22:31:41.833722 3286 tablet_replica.cc:439] T 44f6f034569b473f91f98b770b90de3b P b1cd70f211c04db89dce7456ba0c76a6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } } peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } } }
I20260808 22:31:41.833853 3286 tablet_replica.cc:442] T 44f6f034569b473f91f98b770b90de3b P b1cd70f211c04db89dce7456ba0c76a6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:31:41.929978 3285 raft_consensus.cc:482] T 44f6f034569b473f91f98b770b90de3b P c1071b6848624168ac96b1e5419ae225 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:31:41.930116 3285 raft_consensus.cc:504] T 44f6f034569b473f91f98b770b90de3b P c1071b6848624168ac96b1e5419ae225 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } } peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } }
I20260808 22:31:41.931249 3285 leader_election.cc:288] T 44f6f034569b473f91f98b770b90de3b P c1071b6848624168ac96b1e5419ae225 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1cd70f211c04db89dce7456ba0c76a6 (127.31.87.130:41725), 130f382ce0ca4e189e6cecf96b49ea59 (127.31.87.131:45421)
I20260808 22:31:41.935762 3268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:41.931626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:47771 (local address 127.31.87.131:45421)
0808 22:31:41.931794 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:41.931798 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:41.931912 (+ 114us) server_negotiation.cc:407] Connection header received
0808 22:31:41.932018 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:41.932021 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:41.932065 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:41.932136 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:31:41.932448 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.933144 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.934635 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.935221 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.935343 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:41.935395 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:41.935576 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 22:31:41.935641 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260808 22:31:41.935858 3272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:41.931562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:33883 (local address 127.31.87.130:41725)
0808 22:31:41.931707 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:41.931711 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:41.931890 (+ 179us) server_negotiation.cc:407] Connection header received
0808 22:31:41.932030 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:41.932033 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:41.932072 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:41.932135 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:31:41.932845 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.933543 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.934814 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.935376 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.935479 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:41.935514 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:41.935676 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 22:31:41.935743 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 22:31:41.935788 3305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:41.931482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.131:45421 (local address 127.31.87.129:47771)
0808 22:31:41.931824 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:41.931835 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:41.931932 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:41.932148 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:41.932151 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:41.932166 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:31:41.932331 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:41.932339 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.933234 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.933237 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:41.934511 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:41.934521 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.935361 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.935364 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:41.935448 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:41.935475 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 22:31:41.935545 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 22:31:41.935624 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"mutex_wait_us":25,"thread_start_us":94,"threads_started":1}
I20260808 22:31:41.935808 3306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:41.931478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.130:41725 (local address 127.31.87.129:33883)
0808 22:31:41.931792 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:41.931806 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:41.931916 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:41.932197 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:41.932200 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:41.932216 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:31:41.932730 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:41.932739 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.933647 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.933650 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:41.934700 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:41.934707 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.935474 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.935477 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:41.935565 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:41.935589 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 22:31:41.935652 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 22:31:41.935706 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"mutex_wait_us":1,"thread_start_us":104,"threads_started":1}
I20260808 22:31:41.936487 2910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44f6f034569b473f91f98b770b90de3b" candidate_uuid: "c1071b6848624168ac96b1e5419ae225" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1cd70f211c04db89dce7456ba0c76a6" is_pre_election: true
I20260808 22:31:41.936611 2910 raft_consensus.cc:2454] T 44f6f034569b473f91f98b770b90de3b P b1cd70f211c04db89dce7456ba0c76a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1071b6848624168ac96b1e5419ae225 in term 0.
I20260808 22:31:41.936564 3150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44f6f034569b473f91f98b770b90de3b" candidate_uuid: "c1071b6848624168ac96b1e5419ae225" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130f382ce0ca4e189e6cecf96b49ea59" is_pre_election: true
I20260808 22:31:41.936787 3150 raft_consensus.cc:2454] T 44f6f034569b473f91f98b770b90de3b P 130f382ce0ca4e189e6cecf96b49ea59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1071b6848624168ac96b1e5419ae225 in term 0.
I20260808 22:31:41.936932 2583 leader_election.cc:302] T 44f6f034569b473f91f98b770b90de3b P c1071b6848624168ac96b1e5419ae225 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b1cd70f211c04db89dce7456ba0c76a6, c1071b6848624168ac96b1e5419ae225; no voters:
I20260808 22:31:41.937247 3285 raft_consensus.cc:2781] T 44f6f034569b473f91f98b770b90de3b P c1071b6848624168ac96b1e5419ae225 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:31:41.937317 3285 raft_consensus.cc:482] T 44f6f034569b473f91f98b770b90de3b P c1071b6848624168ac96b1e5419ae225 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:31:41.937345 3285 raft_consensus.cc:3037] T 44f6f034569b473f91f98b770b90de3b P c1071b6848624168ac96b1e5419ae225 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:41.938299 3285 raft_consensus.cc:504] T 44f6f034569b473f91f98b770b90de3b P c1071b6848624168ac96b1e5419ae225 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } } peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } }
I20260808 22:31:41.938601 3285 leader_election.cc:288] T 44f6f034569b473f91f98b770b90de3b P c1071b6848624168ac96b1e5419ae225 [CANDIDATE]: Term 1 election: Requested vote from peers b1cd70f211c04db89dce7456ba0c76a6 (127.31.87.130:41725), 130f382ce0ca4e189e6cecf96b49ea59 (127.31.87.131:45421)
I20260808 22:31:41.938882 2910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44f6f034569b473f91f98b770b90de3b" candidate_uuid: "c1071b6848624168ac96b1e5419ae225" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1cd70f211c04db89dce7456ba0c76a6"
I20260808 22:31:41.938899 3150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44f6f034569b473f91f98b770b90de3b" candidate_uuid: "c1071b6848624168ac96b1e5419ae225" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130f382ce0ca4e189e6cecf96b49ea59"
I20260808 22:31:41.938988 3150 raft_consensus.cc:3037] T 44f6f034569b473f91f98b770b90de3b P 130f382ce0ca4e189e6cecf96b49ea59 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:41.938987 2910 raft_consensus.cc:3037] T 44f6f034569b473f91f98b770b90de3b P b1cd70f211c04db89dce7456ba0c76a6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:41.939911 2910 raft_consensus.cc:2454] T 44f6f034569b473f91f98b770b90de3b P b1cd70f211c04db89dce7456ba0c76a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1071b6848624168ac96b1e5419ae225 in term 1.
I20260808 22:31:41.940001 3150 raft_consensus.cc:2454] T 44f6f034569b473f91f98b770b90de3b P 130f382ce0ca4e189e6cecf96b49ea59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1071b6848624168ac96b1e5419ae225 in term 1.
I20260808 22:31:41.940152 2583 leader_election.cc:302] T 44f6f034569b473f91f98b770b90de3b P c1071b6848624168ac96b1e5419ae225 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b1cd70f211c04db89dce7456ba0c76a6, c1071b6848624168ac96b1e5419ae225; no voters:
I20260808 22:31:41.940337 3285 raft_consensus.cc:2781] T 44f6f034569b473f91f98b770b90de3b P c1071b6848624168ac96b1e5419ae225 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:31:41.940521 3285 raft_consensus.cc:686] T 44f6f034569b473f91f98b770b90de3b P c1071b6848624168ac96b1e5419ae225 [term 1 LEADER]: Becoming Leader. State: Replica: c1071b6848624168ac96b1e5419ae225, State: Running, Role: LEADER
I20260808 22:31:41.940806 3285 consensus_queue.cc:237] T 44f6f034569b473f91f98b770b90de3b P c1071b6848624168ac96b1e5419ae225 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } } peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } }
I20260808 22:31:41.942978 3307 tablet_replica.cc:439] T 44f6f034569b473f91f98b770b90de3b P c1071b6848624168ac96b1e5419ae225: TxnStatusTablet state changed. Reason: New leader c1071b6848624168ac96b1e5419ae225. Latest consensus state: current_term: 1 leader_uuid: "c1071b6848624168ac96b1e5419ae225" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } } peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } } }
I20260808 22:31:41.943069 3307 tablet_replica.cc:442] T 44f6f034569b473f91f98b770b90de3b P c1071b6848624168ac96b1e5419ae225: This TxnStatusTablet replica's current role is: LEADER
I20260808 22:31:41.943323 3310 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 22:31:41.943509 3310 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 22:31:41.944231 2433 catalog_manager.cc:5495] T 44f6f034569b473f91f98b770b90de3b P c1071b6848624168ac96b1e5419ae225 reported cstate change: term changed from 0 to 1, leader changed from <none> to c1071b6848624168ac96b1e5419ae225 (127.31.87.129). New cstate: current_term: 1 leader_uuid: "c1071b6848624168ac96b1e5419ae225" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } health_report { overall_health: UNKNOWN } } }
I20260808 22:31:41.956667 3269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:41.953174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33826 (local address 127.31.87.129:36811)
0808 22:31:41.953272 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:41.953276 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:41.953288 (+ 12us) server_negotiation.cc:407] Connection header received
0808 22:31:41.953466 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:41.953469 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:41.953519 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:41.953595 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:31:41.953829 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.954300 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.955084 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.955301 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.955405 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:41.955727 (+ 322us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:31:41.956214 (+ 487us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:31:41.956332 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:41.956518 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 22:31:41.956573 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 22:31:41.956699 3300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:41.953057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:36811 (local address 127.0.0.1:33826)
0808 22:31:41.953222 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:41.953237 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:41.953345 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:41.953617 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:41.953620 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:41.953627 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 22:31:41.953736 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:41.953741 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.954396 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.954400 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:41.954983 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:41.954991 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.955394 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.955398 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:41.955499 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:41.955549 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 22:31:41.956378 (+ 829us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 22:31:41.956393 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 22:31:41.956477 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 22:31:41.956566 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260808 22:31:41.961310 3150 raft_consensus.cc:1261] T 44f6f034569b473f91f98b770b90de3b P 130f382ce0ca4e189e6cecf96b49ea59 [term 1 FOLLOWER]: Refusing update from remote peer c1071b6848624168ac96b1e5419ae225: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:31:41.961390 2910 raft_consensus.cc:1261] T 44f6f034569b473f91f98b770b90de3b P b1cd70f211c04db89dce7456ba0c76a6 [term 1 FOLLOWER]: Refusing update from remote peer c1071b6848624168ac96b1e5419ae225: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:31:41.961855 3307 consensus_queue.cc:1035] T 44f6f034569b473f91f98b770b90de3b P c1071b6848624168ac96b1e5419ae225 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:31:41.962013 3285 consensus_queue.cc:1035] T 44f6f034569b473f91f98b770b90de3b P c1071b6848624168ac96b1e5419ae225 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:31:41.965329 3286 tablet_replica.cc:439] T 44f6f034569b473f91f98b770b90de3b P b1cd70f211c04db89dce7456ba0c76a6: TxnStatusTablet state changed. Reason: New leader c1071b6848624168ac96b1e5419ae225. Latest consensus state: current_term: 1 leader_uuid: "c1071b6848624168ac96b1e5419ae225" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } } peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } } }
I20260808 22:31:41.965436 3286 tablet_replica.cc:442] T 44f6f034569b473f91f98b770b90de3b P b1cd70f211c04db89dce7456ba0c76a6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:31:41.966121 3284 tablet_replica.cc:439] T 44f6f034569b473f91f98b770b90de3b P 130f382ce0ca4e189e6cecf96b49ea59: TxnStatusTablet state changed. Reason: New leader c1071b6848624168ac96b1e5419ae225. Latest consensus state: current_term: 1 leader_uuid: "c1071b6848624168ac96b1e5419ae225" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } } peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } } }
I20260808 22:31:41.966231 3284 tablet_replica.cc:442] T 44f6f034569b473f91f98b770b90de3b P 130f382ce0ca4e189e6cecf96b49ea59: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:31:41.967641 3285 tablet_replica.cc:439] T 44f6f034569b473f91f98b770b90de3b P c1071b6848624168ac96b1e5419ae225: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c1071b6848624168ac96b1e5419ae225" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } } peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } } }
I20260808 22:31:41.967741 3285 tablet_replica.cc:442] T 44f6f034569b473f91f98b770b90de3b P c1071b6848624168ac96b1e5419ae225: This TxnStatusTablet replica's current role is: LEADER
I20260808 22:31:41.968236 3313 mvcc.cc:205] Tried to move back new op lower bound from 7316391124827357184 to 7316391124752220160. Current Snapshot: MvccSnapshot[applied={T|T < 7316391124827357184}]
I20260808 22:31:41.968534 3307 tablet_replica.cc:439] T 44f6f034569b473f91f98b770b90de3b P c1071b6848624168ac96b1e5419ae225: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c1071b6848624168ac96b1e5419ae225" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } } peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } } }
I20260808 22:31:41.968623 3307 tablet_replica.cc:442] T 44f6f034569b473f91f98b770b90de3b P c1071b6848624168ac96b1e5419ae225: This TxnStatusTablet replica's current role is: LEADER
I20260808 22:31:41.968842 3284 tablet_replica.cc:439] T 44f6f034569b473f91f98b770b90de3b P 130f382ce0ca4e189e6cecf96b49ea59: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c1071b6848624168ac96b1e5419ae225" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } } peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } } }
I20260808 22:31:41.968940 3284 tablet_replica.cc:442] T 44f6f034569b473f91f98b770b90de3b P 130f382ce0ca4e189e6cecf96b49ea59: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:31:41.969726 3286 tablet_replica.cc:439] T 44f6f034569b473f91f98b770b90de3b P b1cd70f211c04db89dce7456ba0c76a6: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c1071b6848624168ac96b1e5419ae225" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } } peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } } }
I20260808 22:31:41.969826 3286 tablet_replica.cc:442] T 44f6f034569b473f91f98b770b90de3b P b1cd70f211c04db89dce7456ba0c76a6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:31:41.971581 3315 mvcc.cc:205] Tried to move back new op lower bound from 7316391124827357184 to 7316391124752220160. Current Snapshot: MvccSnapshot[applied={T|T < 7316391124827357184}]
I20260808 22:31:41.981660 3268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:41.978048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34590 (local address 127.31.87.131:45421)
0808 22:31:41.978151 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:41.978154 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:41.978182 (+ 28us) server_negotiation.cc:407] Connection header received
0808 22:31:41.978336 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:41.978338 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:41.978386 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:41.978463 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:31:41.978695 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.979165 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.980305 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.980498 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.980560 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:41.980823 (+ 263us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:31:41.981241 (+ 418us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:31:41.981348 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:41.981489 (+ 141us) server_negotiation.cc:299] Negotiation successful
0808 22:31:41.981537 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 22:31:41.991219 3246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:41.988194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:36811 (local address 127.31.87.131:56187)
0808 22:31:41.988404 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:41.988417 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:41.988497 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:41.988704 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:41.988707 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:41.988713 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 22:31:41.988835 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:41.988840 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.989553 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.989556 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:41.990175 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:41.990183 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.990585 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.990589 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:41.990676 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:41.990715 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 22:31:41.990979 (+ 264us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 22:31:41.990986 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 22:31:41.991042 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 22:31:41.991110 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122}
I20260808 22:31:41.991219 3269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:41.988288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:56187 (local address 127.31.87.129:36811)
0808 22:31:41.988398 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:41.988402 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:41.988455 (+ 53us) server_negotiation.cc:407] Connection header received
0808 22:31:41.988586 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:41.988588 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:41.988629 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:41.988695 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:31:41.988926 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.989461 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.990279 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:41.990486 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:41.990565 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:41.990847 (+ 282us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:31:41.990909 (+ 62us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:31:41.990966 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:41.991051 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 22:31:41.991099 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 22:31:41.997704 2910 raft_consensus.cc:1261] T 362f2fe32f2b492a967b8a3718f5ef8f P b1cd70f211c04db89dce7456ba0c76a6 [term 1 FOLLOWER]: Refusing update from remote peer 130f382ce0ca4e189e6cecf96b49ea59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:31:41.997887 2683 raft_consensus.cc:1261] T 362f2fe32f2b492a967b8a3718f5ef8f P c1071b6848624168ac96b1e5419ae225 [term 1 FOLLOWER]: Refusing update from remote peer 130f382ce0ca4e189e6cecf96b49ea59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:31:41.998111 3290 consensus_queue.cc:1035] T 362f2fe32f2b492a967b8a3718f5ef8f P 130f382ce0ca4e189e6cecf96b49ea59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:31:41.998368 3284 consensus_queue.cc:1035] T 362f2fe32f2b492a967b8a3718f5ef8f P 130f382ce0ca4e189e6cecf96b49ea59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:31:42.001287 3314 mvcc.cc:205] Tried to move back new op lower bound from 7316391124978597888 to 7316391123980931072. Current Snapshot: MvccSnapshot[applied={T|T < 7316391124978597888}]
I20260808 22:31:42.020051 3268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:42.016381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:60401 (local address 127.31.87.131:45421)
0808 22:31:42.016484 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:42.016489 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:42.016757 (+ 268us) server_negotiation.cc:407] Connection header received
0808 22:31:42.016883 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:42.016886 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:42.016933 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:42.017002 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:31:42.017443 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:42.018160 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:42.018988 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:42.019223 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:42.019307 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:42.019590 (+ 283us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:31:42.019669 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:31:42.019736 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:42.019867 (+ 131us) server_negotiation.cc:299] Negotiation successful
0808 22:31:42.019924 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 22:31:42.020084 3336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:42.016328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.131:45421 (local address 127.31.87.129:60401)
0808 22:31:42.016657 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:42.016671 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:42.016794 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:42.017034 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:42.017037 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:42.017046 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 22:31:42.017210 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:42.017218 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:42.018253 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:42.018256 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:42.018883 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:42.018891 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:42.019352 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:42.019355 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:42.019435 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:42.019477 (+ 42us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 22:31:42.019760 (+ 283us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 22:31:42.019768 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 22:31:42.019843 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 22:31:42.019925 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":119,"threads_started":1}
I20260808 22:31:42.043721 2432 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60648:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:31:42.043888 2432 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60648:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:31:42.046257 2432 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 256a7c3ccaf0495a895c0b2eab960713: Sending DeleteTablet for 3 replicas of tablet 362f2fe32f2b492a967b8a3718f5ef8f
I20260808 22:31:42.046864 3119 tablet_service.cc:1514] Processing DeleteTablet for tablet 362f2fe32f2b492a967b8a3718f5ef8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:31:42 UTC) from {username='kudu', principal='kudu/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:34584
I20260808 22:31:42.047305 2871 tablet_service.cc:1514] Processing DeleteTablet for tablet 362f2fe32f2b492a967b8a3718f5ef8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:31:42 UTC) from {username='kudu', principal='kudu/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:43404
I20260808 22:31:42.047894 2651 tablet_service.cc:1514] Processing DeleteTablet for tablet 362f2fe32f2b492a967b8a3718f5ef8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:31:42 UTC) from {username='kudu', principal='kudu/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:33816
I20260808 22:31:42.048038 2432 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60648:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:31:42.048434 2432 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:31:42.048668 3344 tablet_replica.cc:330] T 362f2fe32f2b492a967b8a3718f5ef8f P c1071b6848624168ac96b1e5419ae225: stopping tablet replica
I20260808 22:31:42.048799 3343 tablet_replica.cc:330] T 362f2fe32f2b492a967b8a3718f5ef8f P b1cd70f211c04db89dce7456ba0c76a6: stopping tablet replica
I20260808 22:31:42.049362 3343 raft_consensus.cc:2229] T 362f2fe32f2b492a967b8a3718f5ef8f P b1cd70f211c04db89dce7456ba0c76a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:31:42.049538 3344 raft_consensus.cc:2229] T 362f2fe32f2b492a967b8a3718f5ef8f P c1071b6848624168ac96b1e5419ae225 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:31:42.049805 3343 raft_consensus.cc:2258] T 362f2fe32f2b492a967b8a3718f5ef8f P b1cd70f211c04db89dce7456ba0c76a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:31:42.050009 3344 raft_consensus.cc:2258] T 362f2fe32f2b492a967b8a3718f5ef8f P c1071b6848624168ac96b1e5419ae225 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:31:42.052878 3344 ts_tablet_manager.cc:1905] T 362f2fe32f2b492a967b8a3718f5ef8f P c1071b6848624168ac96b1e5419ae225: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:31:42.053509 2871 tablet_service.cc:1467] Processing CreateTablet for tablet 5d044bd5d7e047e58704552379c4d2f5 (DEFAULT_TABLE table=test-table [id=605810479bea45e3873405069e8e9b68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:31:42.053800 2871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d044bd5d7e047e58704552379c4d2f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:42.055647 3276 tablet_bootstrap.cc:492] T 5d044bd5d7e047e58704552379c4d2f5 P b1cd70f211c04db89dce7456ba0c76a6: Bootstrap starting.
I20260808 22:31:42.056196 3119 tablet_service.cc:1467] Processing CreateTablet for tablet 5d044bd5d7e047e58704552379c4d2f5 (DEFAULT_TABLE table=test-table [id=605810479bea45e3873405069e8e9b68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:31:42.056455 3119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d044bd5d7e047e58704552379c4d2f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:42.056618 3276 tablet_bootstrap.cc:654] T 5d044bd5d7e047e58704552379c4d2f5 P b1cd70f211c04db89dce7456ba0c76a6: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:42.058395 3276 tablet_bootstrap.cc:492] T 5d044bd5d7e047e58704552379c4d2f5 P b1cd70f211c04db89dce7456ba0c76a6: No bootstrap required, opened a new log
I20260808 22:31:42.058472 3276 ts_tablet_manager.cc:1397] T 5d044bd5d7e047e58704552379c4d2f5 P b1cd70f211c04db89dce7456ba0c76a6: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 22:31:42.058879 3276 raft_consensus.cc:348] T 5d044bd5d7e047e58704552379c4d2f5 P b1cd70f211c04db89dce7456ba0c76a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } } peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } }
I20260808 22:31:42.058979 3276 raft_consensus.cc:374] T 5d044bd5d7e047e58704552379c4d2f5 P b1cd70f211c04db89dce7456ba0c76a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:42.059010 3276 raft_consensus.cc:729] T 5d044bd5d7e047e58704552379c4d2f5 P b1cd70f211c04db89dce7456ba0c76a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1cd70f211c04db89dce7456ba0c76a6, State: Initialized, Role: FOLLOWER
I20260808 22:31:42.059134 3276 consensus_queue.cc:260] T 5d044bd5d7e047e58704552379c4d2f5 P b1cd70f211c04db89dce7456ba0c76a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } } peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } }
I20260808 22:31:42.059177 3275 tablet_bootstrap.cc:492] T 5d044bd5d7e047e58704552379c4d2f5 P 130f382ce0ca4e189e6cecf96b49ea59: Bootstrap starting.
I20260808 22:31:42.059402 3276 ts_tablet_manager.cc:1428] T 5d044bd5d7e047e58704552379c4d2f5 P b1cd70f211c04db89dce7456ba0c76a6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 22:31:42.060066 3275 tablet_bootstrap.cc:654] T 5d044bd5d7e047e58704552379c4d2f5 P 130f382ce0ca4e189e6cecf96b49ea59: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:42.060293 3342 tablet_replica.cc:330] T 362f2fe32f2b492a967b8a3718f5ef8f P 130f382ce0ca4e189e6cecf96b49ea59: stopping tablet replica
I20260808 22:31:42.060786 3342 raft_consensus.cc:2229] T 362f2fe32f2b492a967b8a3718f5ef8f P 130f382ce0ca4e189e6cecf96b49ea59 [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:31:42.061260 3343 ts_tablet_manager.cc:1905] T 362f2fe32f2b492a967b8a3718f5ef8f P b1cd70f211c04db89dce7456ba0c76a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:31:42.061646 3342 raft_consensus.cc:2258] T 362f2fe32f2b492a967b8a3718f5ef8f P 130f382ce0ca4e189e6cecf96b49ea59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:31:42.063886 3343 ts_tablet_manager.cc:1918] T 362f2fe32f2b492a967b8a3718f5ef8f P b1cd70f211c04db89dce7456ba0c76a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 22:31:42.063969 3343 log.cc:1196] T 362f2fe32f2b492a967b8a3718f5ef8f P b1cd70f211c04db89dce7456ba0c76a6: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-1/wal/wals/362f2fe32f2b492a967b8a3718f5ef8f
I20260808 22:31:42.053186 2651 tablet_service.cc:1467] Processing CreateTablet for tablet 5d044bd5d7e047e58704552379c4d2f5 (DEFAULT_TABLE table=test-table [id=605810479bea45e3873405069e8e9b68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:31:42.064254 3343 ts_tablet_manager.cc:1939] T 362f2fe32f2b492a967b8a3718f5ef8f P b1cd70f211c04db89dce7456ba0c76a6: Deleting consensus metadata
I20260808 22:31:42.064288 2651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d044bd5d7e047e58704552379c4d2f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:42.065362 2411 catalog_manager.cc:4841] TS b1cd70f211c04db89dce7456ba0c76a6 (127.31.87.130:41725): tablet 362f2fe32f2b492a967b8a3718f5ef8f (table test-table [id=4301b3d9d08e4432b30dbff02c40ece0]) successfully deleted
I20260808 22:31:42.066445 3277 tablet_bootstrap.cc:492] T 5d044bd5d7e047e58704552379c4d2f5 P c1071b6848624168ac96b1e5419ae225: Bootstrap starting.
I20260808 22:31:42.067355 3277 tablet_bootstrap.cc:654] T 5d044bd5d7e047e58704552379c4d2f5 P c1071b6848624168ac96b1e5419ae225: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:42.069622 3344 ts_tablet_manager.cc:1918] T 362f2fe32f2b492a967b8a3718f5ef8f P c1071b6848624168ac96b1e5419ae225: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 22:31:42.069753 3344 log.cc:1196] T 362f2fe32f2b492a967b8a3718f5ef8f P c1071b6848624168ac96b1e5419ae225: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-0/wal/wals/362f2fe32f2b492a967b8a3718f5ef8f
I20260808 22:31:42.069962 3277 tablet_bootstrap.cc:492] T 5d044bd5d7e047e58704552379c4d2f5 P c1071b6848624168ac96b1e5419ae225: No bootstrap required, opened a new log
I20260808 22:31:42.070044 3277 ts_tablet_manager.cc:1397] T 5d044bd5d7e047e58704552379c4d2f5 P c1071b6848624168ac96b1e5419ae225: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260808 22:31:42.070142 3342 ts_tablet_manager.cc:1905] T 362f2fe32f2b492a967b8a3718f5ef8f P 130f382ce0ca4e189e6cecf96b49ea59: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:31:42.070287 3344 ts_tablet_manager.cc:1939] T 362f2fe32f2b492a967b8a3718f5ef8f P c1071b6848624168ac96b1e5419ae225: Deleting consensus metadata
I20260808 22:31:42.070466 3277 raft_consensus.cc:348] T 5d044bd5d7e047e58704552379c4d2f5 P c1071b6848624168ac96b1e5419ae225 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } } peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } }
I20260808 22:31:42.070568 3277 raft_consensus.cc:374] T 5d044bd5d7e047e58704552379c4d2f5 P c1071b6848624168ac96b1e5419ae225 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:42.070602 3277 raft_consensus.cc:729] T 5d044bd5d7e047e58704552379c4d2f5 P c1071b6848624168ac96b1e5419ae225 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1071b6848624168ac96b1e5419ae225, State: Initialized, Role: FOLLOWER
I20260808 22:31:42.070719 3277 consensus_queue.cc:260] T 5d044bd5d7e047e58704552379c4d2f5 P c1071b6848624168ac96b1e5419ae225 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } } peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } }
I20260808 22:31:42.070986 3277 ts_tablet_manager.cc:1428] T 5d044bd5d7e047e58704552379c4d2f5 P c1071b6848624168ac96b1e5419ae225: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 22:31:42.072288 2410 catalog_manager.cc:4841] TS c1071b6848624168ac96b1e5419ae225 (127.31.87.129:36811): tablet 362f2fe32f2b492a967b8a3718f5ef8f (table test-table [id=4301b3d9d08e4432b30dbff02c40ece0]) successfully deleted
I20260808 22:31:42.072862 3342 ts_tablet_manager.cc:1918] T 362f2fe32f2b492a967b8a3718f5ef8f P 130f382ce0ca4e189e6cecf96b49ea59: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 22:31:42.072962 3342 log.cc:1196] T 362f2fe32f2b492a967b8a3718f5ef8f P 130f382ce0ca4e189e6cecf96b49ea59: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-2/wal/wals/362f2fe32f2b492a967b8a3718f5ef8f
I20260808 22:31:42.073304 3342 ts_tablet_manager.cc:1939] T 362f2fe32f2b492a967b8a3718f5ef8f P 130f382ce0ca4e189e6cecf96b49ea59: Deleting consensus metadata
I20260808 22:31:42.073740 3275 tablet_bootstrap.cc:492] T 5d044bd5d7e047e58704552379c4d2f5 P 130f382ce0ca4e189e6cecf96b49ea59: No bootstrap required, opened a new log
I20260808 22:31:42.073832 3275 ts_tablet_manager.cc:1397] T 5d044bd5d7e047e58704552379c4d2f5 P 130f382ce0ca4e189e6cecf96b49ea59: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.000s
I20260808 22:31:42.074252 3275 raft_consensus.cc:348] T 5d044bd5d7e047e58704552379c4d2f5 P 130f382ce0ca4e189e6cecf96b49ea59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } } peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } }
I20260808 22:31:42.074363 3275 raft_consensus.cc:374] T 5d044bd5d7e047e58704552379c4d2f5 P 130f382ce0ca4e189e6cecf96b49ea59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:42.074396 3275 raft_consensus.cc:729] T 5d044bd5d7e047e58704552379c4d2f5 P 130f382ce0ca4e189e6cecf96b49ea59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130f382ce0ca4e189e6cecf96b49ea59, State: Initialized, Role: FOLLOWER
I20260808 22:31:42.074388 2409 catalog_manager.cc:4841] TS 130f382ce0ca4e189e6cecf96b49ea59 (127.31.87.131:45421): tablet 362f2fe32f2b492a967b8a3718f5ef8f (table test-table [id=4301b3d9d08e4432b30dbff02c40ece0]) successfully deleted
I20260808 22:31:42.074497 3275 consensus_queue.cc:260] T 5d044bd5d7e047e58704552379c4d2f5 P 130f382ce0ca4e189e6cecf96b49ea59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } } peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } }
I20260808 22:31:42.074761 3275 ts_tablet_manager.cc:1428] T 5d044bd5d7e047e58704552379c4d2f5 P 130f382ce0ca4e189e6cecf96b49ea59: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 22:31:42.137472 3340 raft_consensus.cc:482] T 5d044bd5d7e047e58704552379c4d2f5 P c1071b6848624168ac96b1e5419ae225 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:31:42.137601 3340 raft_consensus.cc:504] T 5d044bd5d7e047e58704552379c4d2f5 P c1071b6848624168ac96b1e5419ae225 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } } peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } }
I20260808 22:31:42.137995 3340 leader_election.cc:288] T 5d044bd5d7e047e58704552379c4d2f5 P c1071b6848624168ac96b1e5419ae225 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1cd70f211c04db89dce7456ba0c76a6 (127.31.87.130:41725), 130f382ce0ca4e189e6cecf96b49ea59 (127.31.87.131:45421)
I20260808 22:31:42.138397 3150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d044bd5d7e047e58704552379c4d2f5" candidate_uuid: "c1071b6848624168ac96b1e5419ae225" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130f382ce0ca4e189e6cecf96b49ea59" is_pre_election: true
I20260808 22:31:42.138417 2910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d044bd5d7e047e58704552379c4d2f5" candidate_uuid: "c1071b6848624168ac96b1e5419ae225" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1cd70f211c04db89dce7456ba0c76a6" is_pre_election: true
I20260808 22:31:42.138536 3150 raft_consensus.cc:2454] T 5d044bd5d7e047e58704552379c4d2f5 P 130f382ce0ca4e189e6cecf96b49ea59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1071b6848624168ac96b1e5419ae225 in term 0.
I20260808 22:31:42.138574 2910 raft_consensus.cc:2454] T 5d044bd5d7e047e58704552379c4d2f5 P b1cd70f211c04db89dce7456ba0c76a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1071b6848624168ac96b1e5419ae225 in term 0.
I20260808 22:31:42.138891 2581 leader_election.cc:302] T 5d044bd5d7e047e58704552379c4d2f5 P c1071b6848624168ac96b1e5419ae225 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130f382ce0ca4e189e6cecf96b49ea59, c1071b6848624168ac96b1e5419ae225; no voters:
I20260808 22:31:42.139082 3340 raft_consensus.cc:2781] T 5d044bd5d7e047e58704552379c4d2f5 P c1071b6848624168ac96b1e5419ae225 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:31:42.139158 3340 raft_consensus.cc:482] T 5d044bd5d7e047e58704552379c4d2f5 P c1071b6848624168ac96b1e5419ae225 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:31:42.139197 3340 raft_consensus.cc:3037] T 5d044bd5d7e047e58704552379c4d2f5 P c1071b6848624168ac96b1e5419ae225 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:42.140149 3340 raft_consensus.cc:504] T 5d044bd5d7e047e58704552379c4d2f5 P c1071b6848624168ac96b1e5419ae225 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } } peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } }
I20260808 22:31:42.140456 3340 leader_election.cc:288] T 5d044bd5d7e047e58704552379c4d2f5 P c1071b6848624168ac96b1e5419ae225 [CANDIDATE]: Term 1 election: Requested vote from peers b1cd70f211c04db89dce7456ba0c76a6 (127.31.87.130:41725), 130f382ce0ca4e189e6cecf96b49ea59 (127.31.87.131:45421)
I20260808 22:31:42.140779 3150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d044bd5d7e047e58704552379c4d2f5" candidate_uuid: "c1071b6848624168ac96b1e5419ae225" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130f382ce0ca4e189e6cecf96b49ea59"
I20260808 22:31:42.140753 2910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d044bd5d7e047e58704552379c4d2f5" candidate_uuid: "c1071b6848624168ac96b1e5419ae225" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1cd70f211c04db89dce7456ba0c76a6"
I20260808 22:31:42.140884 3150 raft_consensus.cc:3037] T 5d044bd5d7e047e58704552379c4d2f5 P 130f382ce0ca4e189e6cecf96b49ea59 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:42.140889 2910 raft_consensus.cc:3037] T 5d044bd5d7e047e58704552379c4d2f5 P b1cd70f211c04db89dce7456ba0c76a6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:42.141798 3150 raft_consensus.cc:2454] T 5d044bd5d7e047e58704552379c4d2f5 P 130f382ce0ca4e189e6cecf96b49ea59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1071b6848624168ac96b1e5419ae225 in term 1.
I20260808 22:31:42.142036 2910 raft_consensus.cc:2454] T 5d044bd5d7e047e58704552379c4d2f5 P b1cd70f211c04db89dce7456ba0c76a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1071b6848624168ac96b1e5419ae225 in term 1.
I20260808 22:31:42.142046 2581 leader_election.cc:302] T 5d044bd5d7e047e58704552379c4d2f5 P c1071b6848624168ac96b1e5419ae225 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130f382ce0ca4e189e6cecf96b49ea59, c1071b6848624168ac96b1e5419ae225; no voters:
I20260808 22:31:42.142195 3340 raft_consensus.cc:2781] T 5d044bd5d7e047e58704552379c4d2f5 P c1071b6848624168ac96b1e5419ae225 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:31:42.142271 3340 raft_consensus.cc:686] T 5d044bd5d7e047e58704552379c4d2f5 P c1071b6848624168ac96b1e5419ae225 [term 1 LEADER]: Becoming Leader. State: Replica: c1071b6848624168ac96b1e5419ae225, State: Running, Role: LEADER
I20260808 22:31:42.142400 3340 consensus_queue.cc:237] T 5d044bd5d7e047e58704552379c4d2f5 P c1071b6848624168ac96b1e5419ae225 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } } peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } }
I20260808 22:31:42.144040 2432 catalog_manager.cc:5495] T 5d044bd5d7e047e58704552379c4d2f5 P c1071b6848624168ac96b1e5419ae225 reported cstate change: term changed from 0 to 1, leader changed from <none> to c1071b6848624168ac96b1e5419ae225 (127.31.87.129). New cstate: current_term: 1 leader_uuid: "c1071b6848624168ac96b1e5419ae225" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } health_report { overall_health: UNKNOWN } } }
I20260808 22:31:42.166231 3269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:42.163079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33834 (local address 127.31.87.129:36811)
0808 22:31:42.163183 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:42.163187 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:42.163199 (+ 12us) server_negotiation.cc:407] Connection header received
0808 22:31:42.163325 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:42.163327 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:42.163368 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:42.163424 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:31:42.163677 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:42.164158 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:42.165311 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:42.165509 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:42.165580 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:42.165859 (+ 279us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:31:42.165920 (+ 61us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:31:42.165966 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:42.166064 (+ 98us) server_negotiation.cc:299] Negotiation successful
0808 22:31:42.166114 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 22:31:42.174384 3269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:42.171649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:52707 (local address 127.31.87.129:36811)
0808 22:31:42.171743 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:42.171747 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:42.171759 (+ 12us) server_negotiation.cc:407] Connection header received
0808 22:31:42.171867 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:42.171870 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:42.171907 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:42.171964 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:31:42.172143 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:42.172597 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:42.173382 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:42.173590 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:42.173679 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:42.173912 (+ 233us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:31:42.173973 (+ 61us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:31:42.174038 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:42.174176 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 22:31:42.174241 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":45}
I20260808 22:31:42.174475 3336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:42.171593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:36811 (local address 127.31.87.129:52707)
0808 22:31:42.171699 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:42.171713 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:42.171787 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:42.171979 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:42.171982 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:42.171988 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 22:31:42.172059 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:42.172064 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:42.172705 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:42.172708 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:42.173283 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:42.173292 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:42.173687 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:42.173690 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:42.173795 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:42.173818 (+ 23us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 22:31:42.174080 (+ 262us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 22:31:42.174088 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 22:31:42.174171 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 22:31:42.174272 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":18}
I20260808 22:31:42.180173 2910 raft_consensus.cc:1261] T 5d044bd5d7e047e58704552379c4d2f5 P b1cd70f211c04db89dce7456ba0c76a6 [term 1 FOLLOWER]: Refusing update from remote peer c1071b6848624168ac96b1e5419ae225: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:31:42.180188 3150 raft_consensus.cc:1261] T 5d044bd5d7e047e58704552379c4d2f5 P 130f382ce0ca4e189e6cecf96b49ea59 [term 1 FOLLOWER]: Refusing update from remote peer c1071b6848624168ac96b1e5419ae225: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:31:42.180576 3285 consensus_queue.cc:1035] T 5d044bd5d7e047e58704552379c4d2f5 P c1071b6848624168ac96b1e5419ae225 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:31:42.180743 3340 consensus_queue.cc:1035] T 5d044bd5d7e047e58704552379c4d2f5 P c1071b6848624168ac96b1e5419ae225 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 22:31:42.193681 3334 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 22:31:43.746665 2433 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60648:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:31:43.746812 2433 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60648:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:31:43.748972 2433 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 256a7c3ccaf0495a895c0b2eab960713: Sending DeleteTablet for 3 replicas of tablet 5d044bd5d7e047e58704552379c4d2f5
I20260808 22:31:43.749470 2651 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d044bd5d7e047e58704552379c4d2f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:31:43 UTC) from {username='kudu', principal='kudu/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:33816
I20260808 22:31:43.749485 2871 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d044bd5d7e047e58704552379c4d2f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:31:43 UTC) from {username='kudu', principal='kudu/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:43404
I20260808 22:31:43.749619 3119 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d044bd5d7e047e58704552379c4d2f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:31:43 UTC) from {username='kudu', principal='kudu/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:34584
I20260808 22:31:43.749807 3387 tablet_replica.cc:330] T 5d044bd5d7e047e58704552379c4d2f5 P b1cd70f211c04db89dce7456ba0c76a6: stopping tablet replica
I20260808 22:31:43.749972 3390 tablet_replica.cc:330] T 5d044bd5d7e047e58704552379c4d2f5 P 130f382ce0ca4e189e6cecf96b49ea59: stopping tablet replica
I20260808 22:31:43.749992 3387 raft_consensus.cc:2229] T 5d044bd5d7e047e58704552379c4d2f5 P b1cd70f211c04db89dce7456ba0c76a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:31:43.750142 3387 raft_consensus.cc:2258] T 5d044bd5d7e047e58704552379c4d2f5 P b1cd70f211c04db89dce7456ba0c76a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:31:43.750140 3390 raft_consensus.cc:2229] T 5d044bd5d7e047e58704552379c4d2f5 P 130f382ce0ca4e189e6cecf96b49ea59 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:31:43.750245 3390 raft_consensus.cc:2258] T 5d044bd5d7e047e58704552379c4d2f5 P 130f382ce0ca4e189e6cecf96b49ea59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:31:43.749917 3389 tablet_replica.cc:330] T 5d044bd5d7e047e58704552379c4d2f5 P c1071b6848624168ac96b1e5419ae225: stopping tablet replica
I20260808 22:31:43.750722 3389 raft_consensus.cc:2229] T 5d044bd5d7e047e58704552379c4d2f5 P c1071b6848624168ac96b1e5419ae225 [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:31:43.750988 3387 ts_tablet_manager.cc:1905] T 5d044bd5d7e047e58704552379c4d2f5 P b1cd70f211c04db89dce7456ba0c76a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:31:43.751353 3390 ts_tablet_manager.cc:1905] T 5d044bd5d7e047e58704552379c4d2f5 P 130f382ce0ca4e189e6cecf96b49ea59: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:31:43.752049 3389 raft_consensus.cc:2258] T 5d044bd5d7e047e58704552379c4d2f5 P c1071b6848624168ac96b1e5419ae225 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:31:43.752640 3389 ts_tablet_manager.cc:1905] T 5d044bd5d7e047e58704552379c4d2f5 P c1071b6848624168ac96b1e5419ae225: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:31:43.753966 3390 ts_tablet_manager.cc:1918] T 5d044bd5d7e047e58704552379c4d2f5 P 130f382ce0ca4e189e6cecf96b49ea59: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 22:31:43.754051 3390 log.cc:1196] T 5d044bd5d7e047e58704552379c4d2f5 P 130f382ce0ca4e189e6cecf96b49ea59: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-2/wal/wals/5d044bd5d7e047e58704552379c4d2f5
I20260808 22:31:43.754355 3390 ts_tablet_manager.cc:1939] T 5d044bd5d7e047e58704552379c4d2f5 P 130f382ce0ca4e189e6cecf96b49ea59: Deleting consensus metadata
I20260808 22:31:43.755019 2409 catalog_manager.cc:4841] TS 130f382ce0ca4e189e6cecf96b49ea59 (127.31.87.131:45421): tablet 5d044bd5d7e047e58704552379c4d2f5 (table test-table [id=605810479bea45e3873405069e8e9b68]) successfully deleted
I20260808 22:31:43.755250 3389 ts_tablet_manager.cc:1918] T 5d044bd5d7e047e58704552379c4d2f5 P c1071b6848624168ac96b1e5419ae225: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 22:31:43.755321 3389 log.cc:1196] T 5d044bd5d7e047e58704552379c4d2f5 P c1071b6848624168ac96b1e5419ae225: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-0/wal/wals/5d044bd5d7e047e58704552379c4d2f5
I20260808 22:31:43.755434 3387 ts_tablet_manager.cc:1918] T 5d044bd5d7e047e58704552379c4d2f5 P b1cd70f211c04db89dce7456ba0c76a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 22:31:43.755504 3389 ts_tablet_manager.cc:1939] T 5d044bd5d7e047e58704552379c4d2f5 P c1071b6848624168ac96b1e5419ae225: Deleting consensus metadata
I20260808 22:31:43.755502 3387 log.cc:1196] T 5d044bd5d7e047e58704552379c4d2f5 P b1cd70f211c04db89dce7456ba0c76a6: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-1/wal/wals/5d044bd5d7e047e58704552379c4d2f5
I20260808 22:31:43.755738 3387 ts_tablet_manager.cc:1939] T 5d044bd5d7e047e58704552379c4d2f5 P b1cd70f211c04db89dce7456ba0c76a6: Deleting consensus metadata
I20260808 22:31:43.756289 2411 catalog_manager.cc:4841] TS b1cd70f211c04db89dce7456ba0c76a6 (127.31.87.130:41725): tablet 5d044bd5d7e047e58704552379c4d2f5 (table test-table [id=605810479bea45e3873405069e8e9b68]) successfully deleted
I20260808 22:31:43.756872 2410 catalog_manager.cc:4841] TS c1071b6848624168ac96b1e5419ae225 (127.31.87.129:36811): tablet 5d044bd5d7e047e58704552379c4d2f5 (table test-table [id=605810479bea45e3873405069e8e9b68]) successfully deleted
Aug 08 22:31:43 dist-test-slave-266d krb5kdc[2372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228303, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 22:31:43 dist-test-slave-266d krb5kdc[2372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228303, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:31:43.783907 3402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:43.771370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60664 (local address 127.31.87.190:45895)
0808 22:31:43.771654 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:43.771663 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:43.771677 (+ 14us) server_negotiation.cc:407] Connection header received
0808 22:31:43.771803 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:43.771807 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:43.771853 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:43.771926 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:43.772596 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:43.773201 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:43.774020 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:43.774276 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:43.774364 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:43.781074 (+ 6710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:43.781078 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:43.781080 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:43.781161 (+ 81us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:43.782350 (+ 1189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:43.782735 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:43.782739 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:43.782741 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:43.782837 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:43.783081 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:43.783084 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:43.783086 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:43.783265 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:43.783388 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:43.783620 (+ 232us) server_negotiation.cc:299] Negotiation successful
0808 22:31:43.783738 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":96,"threads_started":1}
I20260808 22:31:43.786304 2433 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60664:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:31:43.786732 2433 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:31:43.791215 2651 tablet_service.cc:1467] Processing CreateTablet for tablet 80cab5d06a994b3fabcefc148663bf25 (DEFAULT_TABLE table=test-table [id=57d9c3a8f16b4a89bd90bf9d6d871267]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:31:43.791246 2871 tablet_service.cc:1467] Processing CreateTablet for tablet 80cab5d06a994b3fabcefc148663bf25 (DEFAULT_TABLE table=test-table [id=57d9c3a8f16b4a89bd90bf9d6d871267]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:31:43.791508 2651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80cab5d06a994b3fabcefc148663bf25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:43.791522 2871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80cab5d06a994b3fabcefc148663bf25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:43.791517 3119 tablet_service.cc:1467] Processing CreateTablet for tablet 80cab5d06a994b3fabcefc148663bf25 (DEFAULT_TABLE table=test-table [id=57d9c3a8f16b4a89bd90bf9d6d871267]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:31:43.791754 3119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80cab5d06a994b3fabcefc148663bf25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:43.793944 3405 tablet_bootstrap.cc:492] T 80cab5d06a994b3fabcefc148663bf25 P b1cd70f211c04db89dce7456ba0c76a6: Bootstrap starting.
I20260808 22:31:43.794037 3406 tablet_bootstrap.cc:492] T 80cab5d06a994b3fabcefc148663bf25 P 130f382ce0ca4e189e6cecf96b49ea59: Bootstrap starting.
I20260808 22:31:43.794423 3404 tablet_bootstrap.cc:492] T 80cab5d06a994b3fabcefc148663bf25 P c1071b6848624168ac96b1e5419ae225: Bootstrap starting.
I20260808 22:31:43.794886 3406 tablet_bootstrap.cc:654] T 80cab5d06a994b3fabcefc148663bf25 P 130f382ce0ca4e189e6cecf96b49ea59: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:43.794886 3405 tablet_bootstrap.cc:654] T 80cab5d06a994b3fabcefc148663bf25 P b1cd70f211c04db89dce7456ba0c76a6: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:43.795253 3404 tablet_bootstrap.cc:654] T 80cab5d06a994b3fabcefc148663bf25 P c1071b6848624168ac96b1e5419ae225: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:43.795949 3405 tablet_bootstrap.cc:492] T 80cab5d06a994b3fabcefc148663bf25 P b1cd70f211c04db89dce7456ba0c76a6: No bootstrap required, opened a new log
I20260808 22:31:43.796039 3405 ts_tablet_manager.cc:1397] T 80cab5d06a994b3fabcefc148663bf25 P b1cd70f211c04db89dce7456ba0c76a6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 22:31:43.796358 3406 tablet_bootstrap.cc:492] T 80cab5d06a994b3fabcefc148663bf25 P 130f382ce0ca4e189e6cecf96b49ea59: No bootstrap required, opened a new log
I20260808 22:31:43.796450 3406 ts_tablet_manager.cc:1397] T 80cab5d06a994b3fabcefc148663bf25 P 130f382ce0ca4e189e6cecf96b49ea59: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 22:31:43.796510 3405 raft_consensus.cc:348] T 80cab5d06a994b3fabcefc148663bf25 P b1cd70f211c04db89dce7456ba0c76a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } } peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } }
I20260808 22:31:43.796633 3405 raft_consensus.cc:374] T 80cab5d06a994b3fabcefc148663bf25 P b1cd70f211c04db89dce7456ba0c76a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:43.796671 3405 raft_consensus.cc:729] T 80cab5d06a994b3fabcefc148663bf25 P b1cd70f211c04db89dce7456ba0c76a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1cd70f211c04db89dce7456ba0c76a6, State: Initialized, Role: FOLLOWER
I20260808 22:31:43.796790 3405 consensus_queue.cc:260] T 80cab5d06a994b3fabcefc148663bf25 P b1cd70f211c04db89dce7456ba0c76a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } } peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } }
I20260808 22:31:43.796835 3406 raft_consensus.cc:348] T 80cab5d06a994b3fabcefc148663bf25 P 130f382ce0ca4e189e6cecf96b49ea59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } } peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } }
I20260808 22:31:43.796921 3406 raft_consensus.cc:374] T 80cab5d06a994b3fabcefc148663bf25 P 130f382ce0ca4e189e6cecf96b49ea59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:43.796969 3406 raft_consensus.cc:729] T 80cab5d06a994b3fabcefc148663bf25 P 130f382ce0ca4e189e6cecf96b49ea59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130f382ce0ca4e189e6cecf96b49ea59, State: Initialized, Role: FOLLOWER
I20260808 22:31:43.797132 3406 consensus_queue.cc:260] T 80cab5d06a994b3fabcefc148663bf25 P 130f382ce0ca4e189e6cecf96b49ea59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } } peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } }
I20260808 22:31:43.797247 3405 ts_tablet_manager.cc:1428] T 80cab5d06a994b3fabcefc148663bf25 P b1cd70f211c04db89dce7456ba0c76a6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 22:31:43.797498 3406 ts_tablet_manager.cc:1428] T 80cab5d06a994b3fabcefc148663bf25 P 130f382ce0ca4e189e6cecf96b49ea59: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 22:31:43.797523 3404 tablet_bootstrap.cc:492] T 80cab5d06a994b3fabcefc148663bf25 P c1071b6848624168ac96b1e5419ae225: No bootstrap required, opened a new log
I20260808 22:31:43.797600 3404 ts_tablet_manager.cc:1397] T 80cab5d06a994b3fabcefc148663bf25 P c1071b6848624168ac96b1e5419ae225: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 22:31:43.798036 3404 raft_consensus.cc:348] T 80cab5d06a994b3fabcefc148663bf25 P c1071b6848624168ac96b1e5419ae225 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } } peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } }
I20260808 22:31:43.798164 3404 raft_consensus.cc:374] T 80cab5d06a994b3fabcefc148663bf25 P c1071b6848624168ac96b1e5419ae225 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:43.798216 3404 raft_consensus.cc:729] T 80cab5d06a994b3fabcefc148663bf25 P c1071b6848624168ac96b1e5419ae225 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1071b6848624168ac96b1e5419ae225, State: Initialized, Role: FOLLOWER
I20260808 22:31:43.798338 3404 consensus_queue.cc:260] T 80cab5d06a994b3fabcefc148663bf25 P c1071b6848624168ac96b1e5419ae225 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } } peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } }
I20260808 22:31:43.798568 3404 ts_tablet_manager.cc:1428] T 80cab5d06a994b3fabcefc148663bf25 P c1071b6848624168ac96b1e5419ae225: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 22:31:44.043887 3414 raft_consensus.cc:482] T 80cab5d06a994b3fabcefc148663bf25 P b1cd70f211c04db89dce7456ba0c76a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:31:44.044041 3414 raft_consensus.cc:504] T 80cab5d06a994b3fabcefc148663bf25 P b1cd70f211c04db89dce7456ba0c76a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } } peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } }
I20260808 22:31:44.045179 3414 leader_election.cc:288] T 80cab5d06a994b3fabcefc148663bf25 P b1cd70f211c04db89dce7456ba0c76a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1071b6848624168ac96b1e5419ae225 (127.31.87.129:36811), 130f382ce0ca4e189e6cecf96b49ea59 (127.31.87.131:45421)
I20260808 22:31:44.049007 3424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:44.045445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:60829 (local address 127.31.87.129:36811)
0808 22:31:44.045721 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:44.045730 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:44.045853 (+ 123us) server_negotiation.cc:407] Connection header received
0808 22:31:44.045995 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:44.045998 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:44.046049 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:44.046113 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:31:44.046435 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:44.046956 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:44.048103 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:44.048507 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:44.048610 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:44.048677 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:44.048821 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 22:31:44.048894 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":92,"threads_started":1}
I20260808 22:31:44.049022 3420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:44.045365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:36811 (local address 127.31.87.130:60829)
0808 22:31:44.045732 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:44.045745 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:44.045884 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:44.046148 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:44.046151 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:44.046168 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:31:44.046327 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:44.046333 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:44.047050 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:44.047054 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:44.047977 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:44.047987 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:44.048610 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:44.048614 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:44.048706 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:44.048729 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 22:31:44.048795 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 22:31:44.048861 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"mutex_wait_us":58,"spinlock_wait_cycles":14080,"thread_start_us":133,"threads_started":1}
I20260808 22:31:44.049135 3422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:44.045428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.131:45421 (local address 127.31.87.130:38461)
0808 22:31:44.045826 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:44.045840 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:44.045935 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:44.046178 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:44.046181 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:44.046212 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:31:44.046344 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:44.046349 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:44.047111 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:44.047114 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:44.047969 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:44.047978 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:44.048699 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:44.048702 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:44.048798 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:44.048818 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 22:31:44.048882 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 22:31:44.048948 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"mutex_wait_us":51,"thread_start_us":108,"threads_started":1}
I20260808 22:31:44.049261 3425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:44.045577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:38461 (local address 127.31.87.131:45421)
0808 22:31:44.045900 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:44.045908 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:44.045923 (+ 15us) server_negotiation.cc:407] Connection header received
0808 22:31:44.046034 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:44.046036 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:44.046077 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:44.046158 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:31:44.046448 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:44.046989 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:44.048130 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:44.048590 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:44.048699 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:44.048751 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:44.048908 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 22:31:44.048986 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":112,"threads_started":1}
I20260808 22:31:44.049638 3149 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80cab5d06a994b3fabcefc148663bf25" candidate_uuid: "b1cd70f211c04db89dce7456ba0c76a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130f382ce0ca4e189e6cecf96b49ea59" is_pre_election: true
I20260808 22:31:44.049811 3149 raft_consensus.cc:2454] T 80cab5d06a994b3fabcefc148663bf25 P 130f382ce0ca4e189e6cecf96b49ea59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1cd70f211c04db89dce7456ba0c76a6 in term 0.
I20260808 22:31:44.049892 2683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80cab5d06a994b3fabcefc148663bf25" candidate_uuid: "b1cd70f211c04db89dce7456ba0c76a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1071b6848624168ac96b1e5419ae225" is_pre_election: true
I20260808 22:31:44.050038 2683 raft_consensus.cc:2454] T 80cab5d06a994b3fabcefc148663bf25 P c1071b6848624168ac96b1e5419ae225 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1cd70f211c04db89dce7456ba0c76a6 in term 0.
I20260808 22:31:44.050235 2797 leader_election.cc:302] T 80cab5d06a994b3fabcefc148663bf25 P b1cd70f211c04db89dce7456ba0c76a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130f382ce0ca4e189e6cecf96b49ea59, b1cd70f211c04db89dce7456ba0c76a6; no voters:
I20260808 22:31:44.050464 3414 raft_consensus.cc:2781] T 80cab5d06a994b3fabcefc148663bf25 P b1cd70f211c04db89dce7456ba0c76a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:31:44.050521 3414 raft_consensus.cc:482] T 80cab5d06a994b3fabcefc148663bf25 P b1cd70f211c04db89dce7456ba0c76a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:31:44.050565 3414 raft_consensus.cc:3037] T 80cab5d06a994b3fabcefc148663bf25 P b1cd70f211c04db89dce7456ba0c76a6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:44.051497 3414 raft_consensus.cc:504] T 80cab5d06a994b3fabcefc148663bf25 P b1cd70f211c04db89dce7456ba0c76a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } } peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } }
I20260808 22:31:44.051789 3414 leader_election.cc:288] T 80cab5d06a994b3fabcefc148663bf25 P b1cd70f211c04db89dce7456ba0c76a6 [CANDIDATE]: Term 1 election: Requested vote from peers c1071b6848624168ac96b1e5419ae225 (127.31.87.129:36811), 130f382ce0ca4e189e6cecf96b49ea59 (127.31.87.131:45421)
I20260808 22:31:44.052107 2683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80cab5d06a994b3fabcefc148663bf25" candidate_uuid: "b1cd70f211c04db89dce7456ba0c76a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1071b6848624168ac96b1e5419ae225"
I20260808 22:31:44.052107 3149 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80cab5d06a994b3fabcefc148663bf25" candidate_uuid: "b1cd70f211c04db89dce7456ba0c76a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130f382ce0ca4e189e6cecf96b49ea59"
I20260808 22:31:44.052222 2683 raft_consensus.cc:3037] T 80cab5d06a994b3fabcefc148663bf25 P c1071b6848624168ac96b1e5419ae225 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:44.052223 3149 raft_consensus.cc:3037] T 80cab5d06a994b3fabcefc148663bf25 P 130f382ce0ca4e189e6cecf96b49ea59 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:44.053325 2683 raft_consensus.cc:2454] T 80cab5d06a994b3fabcefc148663bf25 P c1071b6848624168ac96b1e5419ae225 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1cd70f211c04db89dce7456ba0c76a6 in term 1.
I20260808 22:31:44.053325 3149 raft_consensus.cc:2454] T 80cab5d06a994b3fabcefc148663bf25 P 130f382ce0ca4e189e6cecf96b49ea59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1cd70f211c04db89dce7456ba0c76a6 in term 1.
I20260808 22:31:44.053602 2798 leader_election.cc:302] T 80cab5d06a994b3fabcefc148663bf25 P b1cd70f211c04db89dce7456ba0c76a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b1cd70f211c04db89dce7456ba0c76a6, c1071b6848624168ac96b1e5419ae225; no voters:
I20260808 22:31:44.053755 3414 raft_consensus.cc:2781] T 80cab5d06a994b3fabcefc148663bf25 P b1cd70f211c04db89dce7456ba0c76a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:31:44.053944 3414 raft_consensus.cc:686] T 80cab5d06a994b3fabcefc148663bf25 P b1cd70f211c04db89dce7456ba0c76a6 [term 1 LEADER]: Becoming Leader. State: Replica: b1cd70f211c04db89dce7456ba0c76a6, State: Running, Role: LEADER
I20260808 22:31:44.054163 3414 consensus_queue.cc:237] T 80cab5d06a994b3fabcefc148663bf25 P b1cd70f211c04db89dce7456ba0c76a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } } peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } }
I20260808 22:31:44.056656 2437 catalog_manager.cc:5495] T 80cab5d06a994b3fabcefc148663bf25 P b1cd70f211c04db89dce7456ba0c76a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to b1cd70f211c04db89dce7456ba0c76a6 (127.31.87.130). New cstate: current_term: 1 leader_uuid: "b1cd70f211c04db89dce7456ba0c76a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } health_report { overall_health: UNKNOWN } } }
I20260808 22:31:44.095405 3430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:44.091423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43418 (local address 127.31.87.130:41725)
0808 22:31:44.091657 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:44.091664 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:44.091678 (+ 14us) server_negotiation.cc:407] Connection header received
0808 22:31:44.091759 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:44.091762 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:44.091821 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:44.091945 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:31:44.092146 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:44.092971 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:44.093840 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:44.094225 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:44.094320 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:44.094506 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:31:44.095073 (+ 567us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:31:44.095186 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:44.095215 (+ 29us) server_negotiation.cc:299] Negotiation successful
0808 22:31:44.095266 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":91,"threads_started":1}
I20260808 22:31:44.104521 3434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:44.101342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:36811 (local address 127.31.87.130:57147)
0808 22:31:44.101622 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:44.101634 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:44.101729 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:44.101935 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:44.101938 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:44.101946 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 22:31:44.102045 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:44.102050 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:44.102753 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:44.102757 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:44.103418 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:44.103427 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:44.103850 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:44.103854 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:44.103936 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:44.103975 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 22:31:44.104248 (+ 273us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 22:31:44.104255 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 22:31:44.104312 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 22:31:44.104388 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":92,"threads_started":1}
I20260808 22:31:44.104506 3424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:44.101498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:57147 (local address 127.31.87.129:36811)
0808 22:31:44.101586 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:44.101589 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:44.101725 (+ 136us) server_negotiation.cc:407] Connection header received
0808 22:31:44.101809 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:44.101812 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:44.101856 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:44.101914 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:31:44.102159 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:44.102651 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:44.103525 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:44.103745 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:44.103834 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:44.104106 (+ 272us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:31:44.104169 (+ 63us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:31:44.104227 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:44.104332 (+ 105us) server_negotiation.cc:299] Negotiation successful
0808 22:31:44.104383 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 22:31:44.109486 3149 raft_consensus.cc:1261] T 80cab5d06a994b3fabcefc148663bf25 P 130f382ce0ca4e189e6cecf96b49ea59 [term 1 FOLLOWER]: Refusing update from remote peer b1cd70f211c04db89dce7456ba0c76a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:31:44.109530 2683 raft_consensus.cc:1261] T 80cab5d06a994b3fabcefc148663bf25 P c1071b6848624168ac96b1e5419ae225 [term 1 FOLLOWER]: Refusing update from remote peer b1cd70f211c04db89dce7456ba0c76a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:31:44.109894 3426 consensus_queue.cc:1035] T 80cab5d06a994b3fabcefc148663bf25 P b1cd70f211c04db89dce7456ba0c76a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:31:44.110051 3414 consensus_queue.cc:1035] T 80cab5d06a994b3fabcefc148663bf25 P b1cd70f211c04db89dce7456ba0c76a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:31:44.125551 3430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:44.122658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:60431 (local address 127.31.87.130:41725)
0808 22:31:44.122765 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:44.122770 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:44.122995 (+ 225us) server_negotiation.cc:407] Connection header received
0808 22:31:44.123071 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:44.123073 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:44.123119 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:44.123173 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:31:44.123325 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:44.123790 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:44.124630 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:44.124836 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:44.124934 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:44.125152 (+ 218us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:31:44.125231 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:31:44.125300 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:44.125379 (+ 79us) server_negotiation.cc:299] Negotiation successful
0808 22:31:44.125426 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 22:31:44.125568 3442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:44.122606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.130:41725 (local address 127.31.87.129:60431)
0808 22:31:44.122913 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:44.122925 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:44.123011 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:44.123178 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:44.123180 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:44.123184 (+ 4us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 22:31:44.123269 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:44.123272 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:44.123857 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:44.123859 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:44.124516 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:44.124529 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:44.124932 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:44.124935 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:44.124998 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:44.125025 (+ 27us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 22:31:44.125304 (+ 279us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 22:31:44.125311 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 22:31:44.125371 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 22:31:44.125447 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":102,"threads_started":1}
I20260808 22:31:44.146515 2432 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:31:44.146656 2432 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:31:44.148520 2432 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 256a7c3ccaf0495a895c0b2eab960713: Sending DeleteTablet for 3 replicas of tablet 80cab5d06a994b3fabcefc148663bf25
I20260808 22:31:44.149019 2871 tablet_service.cc:1514] Processing DeleteTablet for tablet 80cab5d06a994b3fabcefc148663bf25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:31:44 UTC) from {username='kudu', principal='kudu/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:43404
I20260808 22:31:44.149077 3119 tablet_service.cc:1514] Processing DeleteTablet for tablet 80cab5d06a994b3fabcefc148663bf25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:31:44 UTC) from {username='kudu', principal='kudu/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:34584
I20260808 22:31:44.149201 3387 tablet_replica.cc:330] T 80cab5d06a994b3fabcefc148663bf25 P b1cd70f211c04db89dce7456ba0c76a6: stopping tablet replica
I20260808 22:31:44.149298 3387 raft_consensus.cc:2229] T 80cab5d06a994b3fabcefc148663bf25 P b1cd70f211c04db89dce7456ba0c76a6 [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:31:44.149559 3387 raft_consensus.cc:2258] T 80cab5d06a994b3fabcefc148663bf25 P b1cd70f211c04db89dce7456ba0c76a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:31:44.149566 3390 tablet_replica.cc:330] T 80cab5d06a994b3fabcefc148663bf25 P 130f382ce0ca4e189e6cecf96b49ea59: stopping tablet replica
I20260808 22:31:44.149695 3390 raft_consensus.cc:2229] T 80cab5d06a994b3fabcefc148663bf25 P 130f382ce0ca4e189e6cecf96b49ea59 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:31:44.149685 2432 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60664:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 22:31:44.149794 3390 raft_consensus.cc:2258] T 80cab5d06a994b3fabcefc148663bf25 P 130f382ce0ca4e189e6cecf96b49ea59 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 22:31:44.150056 2432 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:31:44.150128 2651 tablet_service.cc:1514] Processing DeleteTablet for tablet 80cab5d06a994b3fabcefc148663bf25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:31:44 UTC) from {username='kudu', principal='kudu/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:33816
I20260808 22:31:44.150290 3389 tablet_replica.cc:330] T 80cab5d06a994b3fabcefc148663bf25 P c1071b6848624168ac96b1e5419ae225: stopping tablet replica
I20260808 22:31:44.150274 3387 ts_tablet_manager.cc:1905] T 80cab5d06a994b3fabcefc148663bf25 P b1cd70f211c04db89dce7456ba0c76a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:31:44.150384 3389 raft_consensus.cc:2229] T 80cab5d06a994b3fabcefc148663bf25 P c1071b6848624168ac96b1e5419ae225 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:31:44.150475 3389 raft_consensus.cc:2258] T 80cab5d06a994b3fabcefc148663bf25 P c1071b6848624168ac96b1e5419ae225 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:31:44.151242 3389 ts_tablet_manager.cc:1905] T 80cab5d06a994b3fabcefc148663bf25 P c1071b6848624168ac96b1e5419ae225: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:31:44.152359 3390 ts_tablet_manager.cc:1905] T 80cab5d06a994b3fabcefc148663bf25 P 130f382ce0ca4e189e6cecf96b49ea59: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:31:44.152799 3387 ts_tablet_manager.cc:1918] T 80cab5d06a994b3fabcefc148663bf25 P b1cd70f211c04db89dce7456ba0c76a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 22:31:44.152861 3387 log.cc:1196] T 80cab5d06a994b3fabcefc148663bf25 P b1cd70f211c04db89dce7456ba0c76a6: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-1/wal/wals/80cab5d06a994b3fabcefc148663bf25
I20260808 22:31:44.153077 3387 ts_tablet_manager.cc:1939] T 80cab5d06a994b3fabcefc148663bf25 P b1cd70f211c04db89dce7456ba0c76a6: Deleting consensus metadata
I20260808 22:31:44.153687 2411 catalog_manager.cc:4841] TS b1cd70f211c04db89dce7456ba0c76a6 (127.31.87.130:41725): tablet 80cab5d06a994b3fabcefc148663bf25 (table test-table [id=57d9c3a8f16b4a89bd90bf9d6d871267]) successfully deleted
I20260808 22:31:44.153966 3389 ts_tablet_manager.cc:1918] T 80cab5d06a994b3fabcefc148663bf25 P c1071b6848624168ac96b1e5419ae225: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 22:31:44.154033 3389 log.cc:1196] T 80cab5d06a994b3fabcefc148663bf25 P c1071b6848624168ac96b1e5419ae225: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-0/wal/wals/80cab5d06a994b3fabcefc148663bf25
I20260808 22:31:44.154275 3389 ts_tablet_manager.cc:1939] T 80cab5d06a994b3fabcefc148663bf25 P c1071b6848624168ac96b1e5419ae225: Deleting consensus metadata
I20260808 22:31:44.154748 3390 ts_tablet_manager.cc:1918] T 80cab5d06a994b3fabcefc148663bf25 P 130f382ce0ca4e189e6cecf96b49ea59: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 22:31:44.154814 3390 log.cc:1196] T 80cab5d06a994b3fabcefc148663bf25 P 130f382ce0ca4e189e6cecf96b49ea59: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-2/wal/wals/80cab5d06a994b3fabcefc148663bf25
I20260808 22:31:44.154945 2410 catalog_manager.cc:4841] TS c1071b6848624168ac96b1e5419ae225 (127.31.87.129:36811): tablet 80cab5d06a994b3fabcefc148663bf25 (table test-table [id=57d9c3a8f16b4a89bd90bf9d6d871267]) successfully deleted
I20260808 22:31:44.155045 3390 ts_tablet_manager.cc:1939] T 80cab5d06a994b3fabcefc148663bf25 P 130f382ce0ca4e189e6cecf96b49ea59: Deleting consensus metadata
I20260808 22:31:44.155776 2409 catalog_manager.cc:4841] TS 130f382ce0ca4e189e6cecf96b49ea59 (127.31.87.131:45421): tablet 80cab5d06a994b3fabcefc148663bf25 (table test-table [id=57d9c3a8f16b4a89bd90bf9d6d871267]) successfully deleted
I20260808 22:31:44.157618 2651 tablet_service.cc:1467] Processing CreateTablet for tablet 460d1891300547e399cf280097adf504 (DEFAULT_TABLE table=test-table [id=78cb4b629e03497b8fde672d70d795dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:31:44.157857 2651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 460d1891300547e399cf280097adf504. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:44.158103 2871 tablet_service.cc:1467] Processing CreateTablet for tablet 460d1891300547e399cf280097adf504 (DEFAULT_TABLE table=test-table [id=78cb4b629e03497b8fde672d70d795dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:31:44.158097 3119 tablet_service.cc:1467] Processing CreateTablet for tablet 460d1891300547e399cf280097adf504 (DEFAULT_TABLE table=test-table [id=78cb4b629e03497b8fde672d70d795dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:31:44.158336 2871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 460d1891300547e399cf280097adf504. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:44.158336 3119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 460d1891300547e399cf280097adf504. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:44.160483 3405 tablet_bootstrap.cc:492] T 460d1891300547e399cf280097adf504 P b1cd70f211c04db89dce7456ba0c76a6: Bootstrap starting.
I20260808 22:31:44.160568 3404 tablet_bootstrap.cc:492] T 460d1891300547e399cf280097adf504 P c1071b6848624168ac96b1e5419ae225: Bootstrap starting.
I20260808 22:31:44.161350 3404 tablet_bootstrap.cc:654] T 460d1891300547e399cf280097adf504 P c1071b6848624168ac96b1e5419ae225: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:44.161405 3405 tablet_bootstrap.cc:654] T 460d1891300547e399cf280097adf504 P b1cd70f211c04db89dce7456ba0c76a6: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:44.162031 3406 tablet_bootstrap.cc:492] T 460d1891300547e399cf280097adf504 P 130f382ce0ca4e189e6cecf96b49ea59: Bootstrap starting.
I20260808 22:31:44.162802 3406 tablet_bootstrap.cc:654] T 460d1891300547e399cf280097adf504 P 130f382ce0ca4e189e6cecf96b49ea59: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:44.162819 3404 tablet_bootstrap.cc:492] T 460d1891300547e399cf280097adf504 P c1071b6848624168ac96b1e5419ae225: No bootstrap required, opened a new log
I20260808 22:31:44.162899 3404 ts_tablet_manager.cc:1397] T 460d1891300547e399cf280097adf504 P c1071b6848624168ac96b1e5419ae225: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 22:31:44.163282 3404 raft_consensus.cc:348] T 460d1891300547e399cf280097adf504 P c1071b6848624168ac96b1e5419ae225 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } } peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } }
I20260808 22:31:44.163414 3404 raft_consensus.cc:374] T 460d1891300547e399cf280097adf504 P c1071b6848624168ac96b1e5419ae225 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:44.163456 3404 raft_consensus.cc:729] T 460d1891300547e399cf280097adf504 P c1071b6848624168ac96b1e5419ae225 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1071b6848624168ac96b1e5419ae225, State: Initialized, Role: FOLLOWER
I20260808 22:31:44.163583 3404 consensus_queue.cc:260] T 460d1891300547e399cf280097adf504 P c1071b6848624168ac96b1e5419ae225 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } } peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } }
I20260808 22:31:44.163879 3405 tablet_bootstrap.cc:492] T 460d1891300547e399cf280097adf504 P b1cd70f211c04db89dce7456ba0c76a6: No bootstrap required, opened a new log
I20260808 22:31:44.163882 3404 ts_tablet_manager.cc:1428] T 460d1891300547e399cf280097adf504 P c1071b6848624168ac96b1e5419ae225: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 22:31:44.163950 3405 ts_tablet_manager.cc:1397] T 460d1891300547e399cf280097adf504 P b1cd70f211c04db89dce7456ba0c76a6: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 22:31:44.164026 3406 tablet_bootstrap.cc:492] T 460d1891300547e399cf280097adf504 P 130f382ce0ca4e189e6cecf96b49ea59: No bootstrap required, opened a new log
I20260808 22:31:44.164108 3406 ts_tablet_manager.cc:1397] T 460d1891300547e399cf280097adf504 P 130f382ce0ca4e189e6cecf96b49ea59: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 22:31:44.164348 3405 raft_consensus.cc:348] T 460d1891300547e399cf280097adf504 P b1cd70f211c04db89dce7456ba0c76a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } } peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } }
I20260808 22:31:44.164441 3405 raft_consensus.cc:374] T 460d1891300547e399cf280097adf504 P b1cd70f211c04db89dce7456ba0c76a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:44.164469 3405 raft_consensus.cc:729] T 460d1891300547e399cf280097adf504 P b1cd70f211c04db89dce7456ba0c76a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1cd70f211c04db89dce7456ba0c76a6, State: Initialized, Role: FOLLOWER
I20260808 22:31:44.164489 3406 raft_consensus.cc:348] T 460d1891300547e399cf280097adf504 P 130f382ce0ca4e189e6cecf96b49ea59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } } peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } }
I20260808 22:31:44.164618 3406 raft_consensus.cc:374] T 460d1891300547e399cf280097adf504 P 130f382ce0ca4e189e6cecf96b49ea59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:44.164575 3405 consensus_queue.cc:260] T 460d1891300547e399cf280097adf504 P b1cd70f211c04db89dce7456ba0c76a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } } peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } }
I20260808 22:31:44.164657 3406 raft_consensus.cc:729] T 460d1891300547e399cf280097adf504 P 130f382ce0ca4e189e6cecf96b49ea59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130f382ce0ca4e189e6cecf96b49ea59, State: Initialized, Role: FOLLOWER
I20260808 22:31:44.164816 3405 ts_tablet_manager.cc:1428] T 460d1891300547e399cf280097adf504 P b1cd70f211c04db89dce7456ba0c76a6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 22:31:44.164783 3406 consensus_queue.cc:260] T 460d1891300547e399cf280097adf504 P 130f382ce0ca4e189e6cecf96b49ea59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } } peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } }
I20260808 22:31:44.165053 3406 ts_tablet_manager.cc:1428] T 460d1891300547e399cf280097adf504 P 130f382ce0ca4e189e6cecf96b49ea59: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 22:31:44.492139 3414 raft_consensus.cc:482] T 460d1891300547e399cf280097adf504 P b1cd70f211c04db89dce7456ba0c76a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:31:44.492291 3414 raft_consensus.cc:504] T 460d1891300547e399cf280097adf504 P b1cd70f211c04db89dce7456ba0c76a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } } peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } }
I20260808 22:31:44.492731 3414 leader_election.cc:288] T 460d1891300547e399cf280097adf504 P b1cd70f211c04db89dce7456ba0c76a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1071b6848624168ac96b1e5419ae225 (127.31.87.129:36811), 130f382ce0ca4e189e6cecf96b49ea59 (127.31.87.131:45421)
I20260808 22:31:44.493208 3149 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "460d1891300547e399cf280097adf504" candidate_uuid: "b1cd70f211c04db89dce7456ba0c76a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130f382ce0ca4e189e6cecf96b49ea59" is_pre_election: true
I20260808 22:31:44.493227 2683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "460d1891300547e399cf280097adf504" candidate_uuid: "b1cd70f211c04db89dce7456ba0c76a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1071b6848624168ac96b1e5419ae225" is_pre_election: true
I20260808 22:31:44.493355 3149 raft_consensus.cc:2454] T 460d1891300547e399cf280097adf504 P 130f382ce0ca4e189e6cecf96b49ea59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1cd70f211c04db89dce7456ba0c76a6 in term 0.
I20260808 22:31:44.493381 2683 raft_consensus.cc:2454] T 460d1891300547e399cf280097adf504 P c1071b6848624168ac96b1e5419ae225 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1cd70f211c04db89dce7456ba0c76a6 in term 0.
I20260808 22:31:44.493677 2797 leader_election.cc:302] T 460d1891300547e399cf280097adf504 P b1cd70f211c04db89dce7456ba0c76a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130f382ce0ca4e189e6cecf96b49ea59, b1cd70f211c04db89dce7456ba0c76a6; no voters:
I20260808 22:31:44.493861 3414 raft_consensus.cc:2781] T 460d1891300547e399cf280097adf504 P b1cd70f211c04db89dce7456ba0c76a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:31:44.493914 3414 raft_consensus.cc:482] T 460d1891300547e399cf280097adf504 P b1cd70f211c04db89dce7456ba0c76a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:31:44.493952 3414 raft_consensus.cc:3037] T 460d1891300547e399cf280097adf504 P b1cd70f211c04db89dce7456ba0c76a6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:44.494915 3414 raft_consensus.cc:504] T 460d1891300547e399cf280097adf504 P b1cd70f211c04db89dce7456ba0c76a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } } peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } }
I20260808 22:31:44.495232 3414 leader_election.cc:288] T 460d1891300547e399cf280097adf504 P b1cd70f211c04db89dce7456ba0c76a6 [CANDIDATE]: Term 1 election: Requested vote from peers c1071b6848624168ac96b1e5419ae225 (127.31.87.129:36811), 130f382ce0ca4e189e6cecf96b49ea59 (127.31.87.131:45421)
I20260808 22:31:44.495527 2683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "460d1891300547e399cf280097adf504" candidate_uuid: "b1cd70f211c04db89dce7456ba0c76a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1071b6848624168ac96b1e5419ae225"
I20260808 22:31:44.495575 3149 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "460d1891300547e399cf280097adf504" candidate_uuid: "b1cd70f211c04db89dce7456ba0c76a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130f382ce0ca4e189e6cecf96b49ea59"
I20260808 22:31:44.495630 2683 raft_consensus.cc:3037] T 460d1891300547e399cf280097adf504 P c1071b6848624168ac96b1e5419ae225 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:44.495708 3149 raft_consensus.cc:3037] T 460d1891300547e399cf280097adf504 P 130f382ce0ca4e189e6cecf96b49ea59 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:44.496562 2683 raft_consensus.cc:2454] T 460d1891300547e399cf280097adf504 P c1071b6848624168ac96b1e5419ae225 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1cd70f211c04db89dce7456ba0c76a6 in term 1.
I20260808 22:31:44.496699 3149 raft_consensus.cc:2454] T 460d1891300547e399cf280097adf504 P 130f382ce0ca4e189e6cecf96b49ea59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1cd70f211c04db89dce7456ba0c76a6 in term 1.
I20260808 22:31:44.496831 2798 leader_election.cc:302] T 460d1891300547e399cf280097adf504 P b1cd70f211c04db89dce7456ba0c76a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b1cd70f211c04db89dce7456ba0c76a6, c1071b6848624168ac96b1e5419ae225; no voters:
I20260808 22:31:44.497017 3414 raft_consensus.cc:2781] T 460d1891300547e399cf280097adf504 P b1cd70f211c04db89dce7456ba0c76a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:31:44.497120 3414 raft_consensus.cc:686] T 460d1891300547e399cf280097adf504 P b1cd70f211c04db89dce7456ba0c76a6 [term 1 LEADER]: Becoming Leader. State: Replica: b1cd70f211c04db89dce7456ba0c76a6, State: Running, Role: LEADER
I20260808 22:31:44.497258 3414 consensus_queue.cc:237] T 460d1891300547e399cf280097adf504 P b1cd70f211c04db89dce7456ba0c76a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } } peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } }
I20260808 22:31:44.498872 2437 catalog_manager.cc:5495] T 460d1891300547e399cf280097adf504 P b1cd70f211c04db89dce7456ba0c76a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to b1cd70f211c04db89dce7456ba0c76a6 (127.31.87.130). New cstate: current_term: 1 leader_uuid: "b1cd70f211c04db89dce7456ba0c76a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1cd70f211c04db89dce7456ba0c76a6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 } health_report { overall_health: UNKNOWN } } }
I20260808 22:31:44.537124 3149 raft_consensus.cc:1261] T 460d1891300547e399cf280097adf504 P 130f382ce0ca4e189e6cecf96b49ea59 [term 1 FOLLOWER]: Refusing update from remote peer b1cd70f211c04db89dce7456ba0c76a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:31:44.537164 2683 raft_consensus.cc:1261] T 460d1891300547e399cf280097adf504 P c1071b6848624168ac96b1e5419ae225 [term 1 FOLLOWER]: Refusing update from remote peer b1cd70f211c04db89dce7456ba0c76a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:31:44.537468 3426 consensus_queue.cc:1035] T 460d1891300547e399cf280097adf504 P b1cd70f211c04db89dce7456ba0c76a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130f382ce0ca4e189e6cecf96b49ea59" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:31:44.537627 3414 consensus_queue.cc:1035] T 460d1891300547e399cf280097adf504 P b1cd70f211c04db89dce7456ba0c76a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1071b6848624168ac96b1e5419ae225" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 36811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 22:31:44.549144 3360 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 22:31:46.097848 2432 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:31:46.097999 2432 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:31:46.100354 2432 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 256a7c3ccaf0495a895c0b2eab960713: Sending DeleteTablet for 3 replicas of tablet 460d1891300547e399cf280097adf504
I20260808 22:31:46.100883 2650 tablet_service.cc:1514] Processing DeleteTablet for tablet 460d1891300547e399cf280097adf504 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:31:46 UTC) from {username='kudu', principal='kudu/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:33816
I20260808 22:31:46.101025 3119 tablet_service.cc:1514] Processing DeleteTablet for tablet 460d1891300547e399cf280097adf504 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:31:46 UTC) from {username='kudu', principal='kudu/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:34584
I20260808 22:31:46.101035 2871 tablet_service.cc:1514] Processing DeleteTablet for tablet 460d1891300547e399cf280097adf504 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:31:46 UTC) from {username='kudu', principal='kudu/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:43404
I20260808 22:31:46.101254 3488 tablet_replica.cc:330] T 460d1891300547e399cf280097adf504 P c1071b6848624168ac96b1e5419ae225: stopping tablet replica
I20260808 22:31:46.101410 3488 raft_consensus.cc:2229] T 460d1891300547e399cf280097adf504 P c1071b6848624168ac96b1e5419ae225 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:31:46.101433 3489 tablet_replica.cc:330] T 460d1891300547e399cf280097adf504 P 130f382ce0ca4e189e6cecf96b49ea59: stopping tablet replica
I20260808 22:31:46.101516 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 2550
I20260808 22:31:46.101558 3489 raft_consensus.cc:2229] T 460d1891300547e399cf280097adf504 P 130f382ce0ca4e189e6cecf96b49ea59 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:31:46.101744 3489 raft_consensus.cc:2258] T 460d1891300547e399cf280097adf504 P 130f382ce0ca4e189e6cecf96b49ea59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:31:46.107751 3490 tablet_replica.cc:330] T 460d1891300547e399cf280097adf504 P b1cd70f211c04db89dce7456ba0c76a6: stopping tablet replica
I20260808 22:31:46.107926 3490 raft_consensus.cc:2229] T 460d1891300547e399cf280097adf504 P b1cd70f211c04db89dce7456ba0c76a6 [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:31:46.108193 3490 raft_consensus.cc:2258] T 460d1891300547e399cf280097adf504 P b1cd70f211c04db89dce7456ba0c76a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:31:46.109225 3490 ts_tablet_manager.cc:1905] T 460d1891300547e399cf280097adf504 P b1cd70f211c04db89dce7456ba0c76a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:31:46.112023 3490 ts_tablet_manager.cc:1918] T 460d1891300547e399cf280097adf504 P b1cd70f211c04db89dce7456ba0c76a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 22:31:46.112085 3490 log.cc:1196] T 460d1891300547e399cf280097adf504 P b1cd70f211c04db89dce7456ba0c76a6: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-1/wal/wals/460d1891300547e399cf280097adf504
I20260808 22:31:46.112325 3490 ts_tablet_manager.cc:1939] T 460d1891300547e399cf280097adf504 P b1cd70f211c04db89dce7456ba0c76a6: Deleting consensus metadata
I20260808 22:31:46.116946 3489 ts_tablet_manager.cc:1905] T 460d1891300547e399cf280097adf504 P 130f382ce0ca4e189e6cecf96b49ea59: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:31:46.117038 2411 catalog_manager.cc:4841] TS b1cd70f211c04db89dce7456ba0c76a6 (127.31.87.130:41725): tablet 460d1891300547e399cf280097adf504 (table test-table [id=78cb4b629e03497b8fde672d70d795dc]) successfully deleted
I20260808 22:31:46.119590 3489 ts_tablet_manager.cc:1918] T 460d1891300547e399cf280097adf504 P 130f382ce0ca4e189e6cecf96b49ea59: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 22:31:46.119673 3489 log.cc:1196] T 460d1891300547e399cf280097adf504 P 130f382ce0ca4e189e6cecf96b49ea59: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786228291013016-32094-0/minicluster-data/ts-2/wal/wals/460d1891300547e399cf280097adf504
I20260808 22:31:46.119918 3489 ts_tablet_manager.cc:1939] T 460d1891300547e399cf280097adf504 P 130f382ce0ca4e189e6cecf96b49ea59: Deleting consensus metadata
I20260808 22:31:46.120594 2409 catalog_manager.cc:4841] TS 130f382ce0ca4e189e6cecf96b49ea59 (127.31.87.131:45421): tablet 460d1891300547e399cf280097adf504 (table test-table [id=78cb4b629e03497b8fde672d70d795dc]) successfully deleted
W20260808 22:31:46.124568 2410 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.87.129:36811 (error 108)
I20260808 22:31:46.124872 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 2770
I20260808 22:31:46.128324 3492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:46.126996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:36811 (local address 127.0.0.1:33848)
0808 22:31:46.128119 (+ 1123us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:46.128205 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.129:36811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1043,"thread_start_us":950,"threads_started":1}
W20260808 22:31:46.128600 2410 catalog_manager.cc:4584] TS c1071b6848624168ac96b1e5419ae225 (127.31.87.129:36811): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 460d1891300547e399cf280097adf504: Network error: Client connection negotiation failed: client connection to 127.31.87.129:36811: connect: Connection refused (error 111)
I20260808 22:31:46.134114 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 3007
I20260808 22:31:46.147331 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 2390
2026-08-08T22:31:46Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6777 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:31:46 dist-test-slave-266d krb5kdc[3497](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:31:46 dist-test-slave-266d krb5kdc[3497](info): set up 2 sockets
Aug 08 22:31:46 dist-test-slave-266d krb5kdc[3497](info): commencing operation
krb5kdc: starting...
W20260808 22:31:47.301780 32094 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Aug 08 22:31:47 dist-test-slave-266d krb5kdc[3497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228307, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:31:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:31:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.87.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
I20260808 22:31:47.417043 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:46545
--webserver_interface=127.31.87.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:46611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:46545
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:31:47.524966 3520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:47.525223 3520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:47.525264 3520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:47.529106 3520 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:31:47.529158 3520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:47.529181 3520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:47.529206 3520 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:31:47.529222 3520 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:31:47.534236 3520 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:46611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:46545
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:46545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.87.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:47.535122 3520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:47.536026 3520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:47.543967 3529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:47.543972 3533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:47.544028 3520 server_base.cc:1034] running on GCE node
W20260808 22:31:47.544023 3530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:47.544744 3520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:47.545758 3520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:47.546923 3520 hybrid_clock.cc:648] HybridClock initialized: now 1786228307546912 us; error 25 us; skew 500 ppm
Aug 08 22:31:47 dist-test-slave-266d krb5kdc[3497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228307, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:31:47.549376 3520 init.cc:377] Logged in from keytab as kudu/127.31.87.190@KRBTEST.COM (short username kudu)
I20260808 22:31:47.550137 3520 webserver.cc:460] Webserver started at http://127.31.87.190:37365/ using document root <none> and password file <none>
I20260808 22:31:47.550601 3520 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:47.550654 3520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:47.550813 3520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:47.552673 3520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/master-0/data/instance:
uuid: "b3e3cbcbaacb4a64bc9012be45c356b2"
format_stamp: "Formatted at 2026-08-08 22:31:47 on dist-test-slave-266d"
server_key: "702b39a8331d57eeea072536542e5eac"
server_key_iv: "71affcb4a3b1b1ea5c1f508c22a9b199"
server_key_version: "encryptionkey@0"
I20260808 22:31:47.553200 3520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/master-0/wal/instance:
uuid: "b3e3cbcbaacb4a64bc9012be45c356b2"
format_stamp: "Formatted at 2026-08-08 22:31:47 on dist-test-slave-266d"
server_key: "702b39a8331d57eeea072536542e5eac"
server_key_iv: "71affcb4a3b1b1ea5c1f508c22a9b199"
server_key_version: "encryptionkey@0"
I20260808 22:31:47.556476 3520 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 22:31:47.558645 3545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:47.559738 3520 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:31:47.559846 3520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "b3e3cbcbaacb4a64bc9012be45c356b2"
format_stamp: "Formatted at 2026-08-08 22:31:47 on dist-test-slave-266d"
server_key: "702b39a8331d57eeea072536542e5eac"
server_key_iv: "71affcb4a3b1b1ea5c1f508c22a9b199"
server_key_version: "encryptionkey@0"
I20260808 22:31:47.559937 3520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:47.577111 3520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:47.579530 3520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:47.579687 3520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:47.586241 3520 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.190:46545
I20260808 22:31:47.586269 3631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.190:46545 every 8 connection(s)
I20260808 22:31:47.587098 3520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:31:47.589743 3632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:47.592152 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 3520
I20260808 22:31:47.592242 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/master-0/wal/instance
I20260808 22:31:47.592481 32094 external_mini_cluster.cc:1428] Setting key 5a01138219377dc4c02d0f1c7e047486
I20260808 22:31:47.595906 3632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3e3cbcbaacb4a64bc9012be45c356b2: Bootstrap starting.
I20260808 22:31:47.598569 3632 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b3e3cbcbaacb4a64bc9012be45c356b2: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:47.599391 3632 log.cc:824] T 00000000000000000000000000000000 P b3e3cbcbaacb4a64bc9012be45c356b2: Log is configured to *not* fsync() on all Append() calls
I20260808 22:31:47.601380 3632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3e3cbcbaacb4a64bc9012be45c356b2: No bootstrap required, opened a new log
Aug 08 22:31:47 dist-test-slave-266d krb5kdc[3497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228307, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:31:47.605245 3632 raft_consensus.cc:348] T 00000000000000000000000000000000 P b3e3cbcbaacb4a64bc9012be45c356b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3e3cbcbaacb4a64bc9012be45c356b2" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 46545 } }
I20260808 22:31:47.605474 3632 raft_consensus.cc:374] T 00000000000000000000000000000000 P b3e3cbcbaacb4a64bc9012be45c356b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:47.605540 3632 raft_consensus.cc:729] T 00000000000000000000000000000000 P b3e3cbcbaacb4a64bc9012be45c356b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3e3cbcbaacb4a64bc9012be45c356b2, State: Initialized, Role: FOLLOWER
I20260808 22:31:47.605996 3632 consensus_queue.cc:260] T 00000000000000000000000000000000 P b3e3cbcbaacb4a64bc9012be45c356b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3e3cbcbaacb4a64bc9012be45c356b2" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 46545 } }
I20260808 22:31:47.606175 3632 raft_consensus.cc:388] T 00000000000000000000000000000000 P b3e3cbcbaacb4a64bc9012be45c356b2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:31:47.606251 3632 raft_consensus.cc:482] T 00000000000000000000000000000000 P b3e3cbcbaacb4a64bc9012be45c356b2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:31:47.606348 3632 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b3e3cbcbaacb4a64bc9012be45c356b2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:47.607473 3632 raft_consensus.cc:504] T 00000000000000000000000000000000 P b3e3cbcbaacb4a64bc9012be45c356b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3e3cbcbaacb4a64bc9012be45c356b2" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 46545 } }
I20260808 22:31:47.607841 3632 leader_election.cc:302] T 00000000000000000000000000000000 P b3e3cbcbaacb4a64bc9012be45c356b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b3e3cbcbaacb4a64bc9012be45c356b2; no voters:
I20260808 22:31:47.608121 3635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:47.593942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50130 (local address 127.31.87.190:46545)
0808 22:31:47.594351 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:47.594371 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:47.594415 (+ 44us) server_negotiation.cc:407] Connection header received
0808 22:31:47.595255 (+ 840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:47.595278 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:47.595690 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:47.596056 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:47.596719 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:47.598186 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:47.599025 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:47.599721 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:47.599876 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:47.602240 (+ 2364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:47.602247 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:47.602263 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:47.602296 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:47.605058 (+ 2762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:47.605473 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:47.605480 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:47.605485 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:47.605646 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:47.605943 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:47.605947 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:47.605949 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:47.606458 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:47.606697 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:47.606865 (+ 168us) server_negotiation.cc:299] Negotiation successful
0808 22:31:47.607163 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":104,"threads_started":1}
I20260808 22:31:47.608197 3632 leader_election.cc:288] T 00000000000000000000000000000000 P b3e3cbcbaacb4a64bc9012be45c356b2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:31:47.608294 3638 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b3e3cbcbaacb4a64bc9012be45c356b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:31:47.608493 3638 raft_consensus.cc:686] T 00000000000000000000000000000000 P b3e3cbcbaacb4a64bc9012be45c356b2 [term 1 LEADER]: Becoming Leader. State: Replica: b3e3cbcbaacb4a64bc9012be45c356b2, State: Running, Role: LEADER
I20260808 22:31:47.608809 3638 consensus_queue.cc:237] T 00000000000000000000000000000000 P b3e3cbcbaacb4a64bc9012be45c356b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3e3cbcbaacb4a64bc9012be45c356b2" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 46545 } }
I20260808 22:31:47.609474 3632 sys_catalog.cc:564] T 00000000000000000000000000000000 P b3e3cbcbaacb4a64bc9012be45c356b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:31:47.610301 3640 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3e3cbcbaacb4a64bc9012be45c356b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b3e3cbcbaacb4a64bc9012be45c356b2. Latest consensus state: current_term: 1 leader_uuid: "b3e3cbcbaacb4a64bc9012be45c356b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3e3cbcbaacb4a64bc9012be45c356b2" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 46545 } } }
I20260808 22:31:47.610419 3640 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3e3cbcbaacb4a64bc9012be45c356b2 [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:47.611094 3641 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3e3cbcbaacb4a64bc9012be45c356b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b3e3cbcbaacb4a64bc9012be45c356b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3e3cbcbaacb4a64bc9012be45c356b2" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 46545 } } }
I20260808 22:31:47.611709 3641 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3e3cbcbaacb4a64bc9012be45c356b2 [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:47.611804 3653 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:31:47.614403 3653 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:31:47.619673 3653 catalog_manager.cc:1347] Generated new cluster ID: 210827c59ace4bf789eaee8ff3b9fdbf
I20260808 22:31:47.619736 3653 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:31:47.630311 3653 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:31:47.631342 3653 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:31:47.639470 3653 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b3e3cbcbaacb4a64bc9012be45c356b2: Generated new TSK 0
I20260808 22:31:47.640283 3653 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.87.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab.
Entry for principal kudu/127.31.87.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab.
I20260808 22:31:47.676733 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.129:0
--local_ip_for_outbound_sockets=127.31.87.129
--webserver_interface=127.31.87.129
--webserver_port=0
--tserver_master_addrs=127.31.87.190:46545
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab
--principal=kudu/127.31.87.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:46611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:31:47.783394 3671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:47.783674 3671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:47.783732 3671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:47.787667 3671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:47.787720 3671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:47.787773 3671 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.129
I20260808 22:31:47.792758 3671 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:46611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab
--principal=kudu/127.31.87.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.87.129
--webserver_port=0
--tserver_master_addrs=127.31.87.190:46545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3671
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.129
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:47.793766 3671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:47.794692 3671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:47.804032 3683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:47.804127 3680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:47.804248 3671 server_base.cc:1034] running on GCE node
W20260808 22:31:47.804044 3685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:47.804733 3671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:47.805349 3671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:47.806504 3671 hybrid_clock.cc:648] HybridClock initialized: now 1786228307806475 us; error 39 us; skew 500 ppm
Aug 08 22:31:47 dist-test-slave-266d krb5kdc[3497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228307, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:31:47.809033 3671 init.cc:377] Logged in from keytab as kudu/127.31.87.129@KRBTEST.COM (short username kudu)
I20260808 22:31:47.809934 3671 webserver.cc:460] Webserver started at http://127.31.87.129:44431/ using document root <none> and password file <none>
I20260808 22:31:47.810446 3671 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:47.810528 3671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:47.810778 3671 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:47.812808 3671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-0/data/instance:
uuid: "e4375e0a750f41498ed22cc17560d7d2"
format_stamp: "Formatted at 2026-08-08 22:31:47 on dist-test-slave-266d"
server_key: "66981cd1e6d70fdc810cb4f46aa3dde5"
server_key_iv: "27f0adf0e3964809567a8182033066b6"
server_key_version: "encryptionkey@0"
I20260808 22:31:47.813337 3671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-0/wal/instance:
uuid: "e4375e0a750f41498ed22cc17560d7d2"
format_stamp: "Formatted at 2026-08-08 22:31:47 on dist-test-slave-266d"
server_key: "66981cd1e6d70fdc810cb4f46aa3dde5"
server_key_iv: "27f0adf0e3964809567a8182033066b6"
server_key_version: "encryptionkey@0"
I20260808 22:31:47.816763 3671 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.002s
I20260808 22:31:47.819033 3699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:47.820155 3671 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:31:47.820268 3671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-0/wal
uuid: "e4375e0a750f41498ed22cc17560d7d2"
format_stamp: "Formatted at 2026-08-08 22:31:47 on dist-test-slave-266d"
server_key: "66981cd1e6d70fdc810cb4f46aa3dde5"
server_key_iv: "27f0adf0e3964809567a8182033066b6"
server_key_version: "encryptionkey@0"
I20260808 22:31:47.820355 3671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:47.828872 3671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:47.831318 3671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:47.831477 3671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:47.831990 3671 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:31:47.832741 3671 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:31:47.832788 3671 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:47.832836 3671 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:31:47.832865 3671 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:47.841595 3671 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.129:39857
I20260808 22:31:47.841650 3900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.129:39857 every 8 connection(s)
I20260808 22:31:47.842638 3671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
I20260808 22:31:47.852361 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 3671
I20260808 22:31:47.852458 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-0/wal/instance
Aug 08 22:31:47 dist-test-slave-266d krb5kdc[3497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228307, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.129@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:31:47.852743 32094 external_mini_cluster.cc:1428] Setting key 4cb236fbccfd25f6ab269ede4089f7cf
I20260808 22:31:47.856413 3635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:47.846066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:57005 (local address 127.31.87.190:46545)
0808 22:31:47.846253 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:47.846258 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:47.846276 (+ 18us) server_negotiation.cc:407] Connection header received
0808 22:31:47.847641 (+ 1365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:47.847645 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:47.847701 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:47.847804 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:47.849151 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:47.849675 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:47.850725 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:47.850936 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:47.851004 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:47.853183 (+ 2179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:47.853186 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:47.853188 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:47.853211 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:47.854518 (+ 1307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:47.854976 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:47.854984 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:47.854985 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:47.855071 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:47.855380 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:47.855383 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:47.855385 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:47.855548 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:47.855630 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:47.856187 (+ 557us) server_negotiation.cc:299] Negotiation successful
0808 22:31:47.856285 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 22:31:47.857394 3903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:47.845019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:46545 (local address 127.31.87.129:57005)
0808 22:31:47.845904 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:47.845958 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:47.847414 (+ 1456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:47.847981 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:47.847991 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:47.848475 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:47.848999 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:47.849013 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:47.849774 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:47.849777 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:47.850635 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:47.850645 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:47.851036 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:47.851038 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:47.851183 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:47.851640 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:31:47.851655 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:47.853016 (+ 1361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:47.854621 (+ 1605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:47.854626 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:47.854639 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:47.854885 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:47.855160 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:47.855163 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:47.855165 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:47.855295 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:47.855645 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:47.855651 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:47.855920 (+ 269us) client_negotiation.cc:769] Sending connection context
0808 22:31:47.856115 (+ 195us) client_negotiation.cc:240] Negotiation successful
0808 22:31:47.856430 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":677,"thread_start_us":140,"threads_started":1}
I20260808 22:31:47.858445 3901 heartbeater.cc:347] Connected to a master server at 127.31.87.190:46545
I20260808 22:31:47.858649 3901 heartbeater.cc:464] Registering TS with master...
I20260808 22:31:47.859057 3901 heartbeater.cc:511] Master 127.31.87.190:46545 requested a full tablet report, sending...
I20260808 22:31:47.860431 3565 ts_manager.cc:194] Registered new tserver with Master: e4375e0a750f41498ed22cc17560d7d2 (127.31.87.129:39857)
I20260808 22:31:47.861529 3565 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.129@KRBTEST.COM'} at 127.31.87.129:57005
WARNING: no policy specified for kudu/127.31.87.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab.
Entry for principal kudu/127.31.87.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab.
I20260808 22:31:47.877650 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.130:0
--local_ip_for_outbound_sockets=127.31.87.130
--webserver_interface=127.31.87.130
--webserver_port=0
--tserver_master_addrs=127.31.87.190:46545
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab
--principal=kudu/127.31.87.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:46611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:31:47.986869 3907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:47.987118 3907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:47.987159 3907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:47.991081 3907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:47.991143 3907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:47.991197 3907 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.130
I20260808 22:31:47.996292 3907 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:46611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab
--principal=kudu/127.31.87.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.87.130
--webserver_port=0
--tserver_master_addrs=127.31.87.190:46545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3907
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.130
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:47.997280 3907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:47.998226 3907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:48.007647 3914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:48.007642 3916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:48.007642 3912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:48.007990 3907 server_base.cc:1034] running on GCE node
I20260808 22:31:48.008407 3907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:48.009019 3907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:48.010181 3907 hybrid_clock.cc:648] HybridClock initialized: now 1786228308010161 us; error 27 us; skew 500 ppm
Aug 08 22:31:48 dist-test-slave-266d krb5kdc[3497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228308, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:31:48.012867 3907 init.cc:377] Logged in from keytab as kudu/127.31.87.130@KRBTEST.COM (short username kudu)
I20260808 22:31:48.013759 3907 webserver.cc:460] Webserver started at http://127.31.87.130:46305/ using document root <none> and password file <none>
I20260808 22:31:48.014314 3907 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:48.014380 3907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:48.014567 3907 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:48.016597 3907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-1/data/instance:
uuid: "8de5cdfd3efe4080a30fe85c2fe21826"
format_stamp: "Formatted at 2026-08-08 22:31:48 on dist-test-slave-266d"
server_key: "3f632b770147e8ea61eece3a130a0424"
server_key_iv: "5874939b8294d459c388030719a147ee"
server_key_version: "encryptionkey@0"
I20260808 22:31:48.017185 3907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-1/wal/instance:
uuid: "8de5cdfd3efe4080a30fe85c2fe21826"
format_stamp: "Formatted at 2026-08-08 22:31:48 on dist-test-slave-266d"
server_key: "3f632b770147e8ea61eece3a130a0424"
server_key_iv: "5874939b8294d459c388030719a147ee"
server_key_version: "encryptionkey@0"
I20260808 22:31:48.020710 3907 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 22:31:48.023069 3926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:48.024256 3907 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:31:48.024361 3907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-1/wal
uuid: "8de5cdfd3efe4080a30fe85c2fe21826"
format_stamp: "Formatted at 2026-08-08 22:31:48 on dist-test-slave-266d"
server_key: "3f632b770147e8ea61eece3a130a0424"
server_key_iv: "5874939b8294d459c388030719a147ee"
server_key_version: "encryptionkey@0"
I20260808 22:31:48.024452 3907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:48.053243 3907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:48.057237 3907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:48.057442 3907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:48.058172 3907 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:31:48.059218 3907 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:31:48.059283 3907 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:48.059342 3907 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:31:48.059374 3907 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:48.070837 3907 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.130:40187
I20260808 22:31:48.070868 4127 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.130:40187 every 8 connection(s)
I20260808 22:31:48.071985 3907 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
I20260808 22:31:48.073861 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 3907
I20260808 22:31:48.073974 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-1/wal/instance
I20260808 22:31:48.074263 32094 external_mini_cluster.cc:1428] Setting key 1549015d2b6dc2c04bc4e41039202e0e
Aug 08 22:31:48 dist-test-slave-266d krb5kdc[3497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228308, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.130@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.87.131@KRBTEST.COM; defaulting to no policy
I20260808 22:31:48.087472 3635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:48.073926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:42847 (local address 127.31.87.190:46545)
0808 22:31:48.074780 (+ 854us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:48.074785 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:48.075518 (+ 733us) server_negotiation.cc:407] Connection header received
0808 22:31:48.076583 (+ 1065us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:48.076586 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:48.076645 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:48.076726 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:48.078297 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:48.078818 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:48.080335 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:48.080695 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:48.080818 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:48.083520 (+ 2702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:48.083524 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:48.083526 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:48.083550 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:48.085034 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:48.085725 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:48.085729 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:48.085731 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:48.085824 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:48.086196 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:48.086199 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:48.086201 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:48.086422 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:48.086514 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:48.087162 (+ 648us) server_negotiation.cc:299] Negotiation successful
0808 22:31:48.087307 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":750}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.131@KRBTEST.COM" created.
I20260808 22:31:48.088514 4131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:48.074432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:46545 (local address 127.31.87.130:42847)
0808 22:31:48.074967 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:48.075022 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:48.076378 (+ 1356us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:48.077018 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:48.077028 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:48.077516 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:48.078160 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:48.078179 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:48.078939 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:48.078942 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:48.080213 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:48.080225 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:48.080865 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:48.080869 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:48.081070 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:48.081806 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:31:48.081823 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:48.083327 (+ 1504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:48.085213 (+ 1886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:48.085219 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:48.085235 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:48.085605 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:48.085917 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:48.085921 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:48.085923 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:48.086101 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:48.086526 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:48.086533 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:48.086860 (+ 327us) client_negotiation.cc:769] Sending connection context
0808 22:31:48.087120 (+ 260us) client_negotiation.cc:240] Negotiation successful
0808 22:31:48.087476 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":150,"threads_started":1}
I20260808 22:31:48.089836 4128 heartbeater.cc:347] Connected to a master server at 127.31.87.190:46545
I20260808 22:31:48.090063 4128 heartbeater.cc:464] Registering TS with master...
I20260808 22:31:48.090569 4128 heartbeater.cc:511] Master 127.31.87.190:46545 requested a full tablet report, sending...
I20260808 22:31:48.091722 3565 ts_manager.cc:194] Registered new tserver with Master: 8de5cdfd3efe4080a30fe85c2fe21826 (127.31.87.130:40187)
I20260808 22:31:48.092460 3565 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.130@KRBTEST.COM'} at 127.31.87.130:42847
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab.
Entry for principal kudu/127.31.87.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab.
I20260808 22:31:48.107605 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.131:0
--local_ip_for_outbound_sockets=127.31.87.131
--webserver_interface=127.31.87.131
--webserver_port=0
--tserver_master_addrs=127.31.87.190:46545
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab
--principal=kudu/127.31.87.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:46611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:31:48.214764 4135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:48.214978 4135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:48.215016 4135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:48.218822 4135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:48.218865 4135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:48.218917 4135 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.131
I20260808 22:31:48.223794 4135 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:46611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab
--principal=kudu/127.31.87.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.87.131
--webserver_port=0
--tserver_master_addrs=127.31.87.190:46545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4135
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.131
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:48.224651 4135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:48.225519 4135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:48.234540 4141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:48.234563 4143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:48.234530 4151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:48.234781 4135 server_base.cc:1034] running on GCE node
I20260808 22:31:48.235105 4135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:48.235656 4135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:48.236845 4135 hybrid_clock.cc:648] HybridClock initialized: now 1786228308236817 us; error 38 us; skew 500 ppm
Aug 08 22:31:48 dist-test-slave-266d krb5kdc[3497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228308, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:31:48.239368 4135 init.cc:377] Logged in from keytab as kudu/127.31.87.131@KRBTEST.COM (short username kudu)
I20260808 22:31:48.240177 4135 webserver.cc:460] Webserver started at http://127.31.87.131:45385/ using document root <none> and password file <none>
I20260808 22:31:48.240680 4135 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:48.240744 4135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:48.240924 4135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:48.242872 4135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-2/data/instance:
uuid: "222d0d3ff4164749ab5c52e4e289d891"
format_stamp: "Formatted at 2026-08-08 22:31:48 on dist-test-slave-266d"
server_key: "3d83d7e871ba6305d15d34bd77122fc9"
server_key_iv: "825cf96945c2931f4874b23d25d178f4"
server_key_version: "encryptionkey@0"
I20260808 22:31:48.243373 4135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-2/wal/instance:
uuid: "222d0d3ff4164749ab5c52e4e289d891"
format_stamp: "Formatted at 2026-08-08 22:31:48 on dist-test-slave-266d"
server_key: "3d83d7e871ba6305d15d34bd77122fc9"
server_key_iv: "825cf96945c2931f4874b23d25d178f4"
server_key_version: "encryptionkey@0"
I20260808 22:31:48.246726 4135 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260808 22:31:48.248823 4160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:48.249930 4135 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 22:31:48.250031 4135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-2/wal
uuid: "222d0d3ff4164749ab5c52e4e289d891"
format_stamp: "Formatted at 2026-08-08 22:31:48 on dist-test-slave-266d"
server_key: "3d83d7e871ba6305d15d34bd77122fc9"
server_key_iv: "825cf96945c2931f4874b23d25d178f4"
server_key_version: "encryptionkey@0"
I20260808 22:31:48.250116 4135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:48.273995 4135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:48.276473 4135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:48.276641 4135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:48.277187 4135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:31:48.277951 4135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:31:48.277998 4135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:48.278047 4135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:31:48.278076 4135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:48.286242 4135 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.131:33623
I20260808 22:31:48.286275 4338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.131:33623 every 8 connection(s)
I20260808 22:31:48.287043 4135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
I20260808 22:31:48.293478 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 4135
I20260808 22:31:48.293588 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-2/wal/instance
I20260808 22:31:48.293888 32094 external_mini_cluster.cc:1428] Setting key 17a9fdc25b90492ffb771e975d3805e3
Aug 08 22:31:48 dist-test-slave-266d krb5kdc[3497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228308, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.131@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:31:48.299585 3635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:48.288613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:57915 (local address 127.31.87.190:46545)
0808 22:31:48.288772 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:48.288778 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:48.289579 (+ 801us) server_negotiation.cc:407] Connection header received
0808 22:31:48.290770 (+ 1191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:48.290773 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:48.290820 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:48.290887 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:48.291888 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:48.292405 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:48.293488 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:48.293829 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:48.293938 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:48.296418 (+ 2480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:48.296421 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:48.296423 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:48.296446 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:48.297742 (+ 1296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:48.298202 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:48.298205 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:48.298206 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:48.298279 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:48.298587 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:48.298591 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:48.298592 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:48.298759 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:48.298843 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:48.299306 (+ 463us) server_negotiation.cc:299] Negotiation successful
0808 22:31:48.299421 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 22:31:48.300285 4342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:48.288978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:46545 (local address 127.31.87.131:57915)
0808 22:31:48.289437 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:48.289476 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:48.290598 (+ 1122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:48.291010 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:48.291016 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:48.291327 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:48.291745 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:48.291758 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:48.292494 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:48.292496 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:48.293374 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:48.293383 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:48.293971 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:48.293976 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:48.294117 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:48.294688 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:31:48.294708 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:48.296259 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:48.297849 (+ 1590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:48.297855 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:48.297868 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:48.298109 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:48.298361 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:48.298364 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:48.298366 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:48.298494 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:48.298861 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:48.298868 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:48.299089 (+ 221us) client_negotiation.cc:769] Sending connection context
0808 22:31:48.299248 (+ 159us) client_negotiation.cc:240] Negotiation successful
0808 22:31:48.299532 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":119,"threads_started":1}
I20260808 22:31:48.301262 4339 heartbeater.cc:347] Connected to a master server at 127.31.87.190:46545
I20260808 22:31:48.301463 4339 heartbeater.cc:464] Registering TS with master...
I20260808 22:31:48.301884 4339 heartbeater.cc:511] Master 127.31.87.190:46545 requested a full tablet report, sending...
I20260808 22:31:48.302776 3565 ts_manager.cc:194] Registered new tserver with Master: 222d0d3ff4164749ab5c52e4e289d891 (127.31.87.131:33623)
I20260808 22:31:48.303287 3565 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.131@KRBTEST.COM'} at 127.31.87.131:57915
I20260808 22:31:48.307353 32094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 22:31:48.321619 3635 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:31:48.320353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50136 (local address 127.31.87.190:46545)
0808 22:31:48.320490 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:48.320494 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:48.320638 (+ 144us) server_negotiation.cc:407] Connection header received
0808 22:31:48.320733 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:48.320735 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:48.320773 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:48.320835 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:48.321525 (+ 690us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50136: BlockingRecv error: recv got EOF from 127.0.0.1:50136 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 22:31:48.322638 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 3671
I20260808 22:31:48.335457 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 3907
I20260808 22:31:48.346349 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 4135
I20260808 22:31:48.352559 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 3520
2026-08-08T22:31:48Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2202 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786228291013016-32094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:31:48 dist-test-slave-266d krb5kdc[4355](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:31:48 dist-test-slave-266d krb5kdc[4355](info): set up 2 sockets
Aug 08 22:31:48 dist-test-slave-266d krb5kdc[4355](info): commencing operation
krb5kdc: starting...
W20260808 22:31:49.493563 32094 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Aug 08 22:31:49 dist-test-slave-266d krb5kdc[4355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:31:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:31:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.87.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
I20260808 22:31:49.605532 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:35665
--webserver_interface=127.31.87.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:41987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:35665
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:31:49.711172 4371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:49.711406 4371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:49.711458 4371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:49.715368 4371 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:31:49.715416 4371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:49.715442 4371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:49.715464 4371 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:31:49.715492 4371 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:31:49.720399 4371 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:41987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786228291013016-32094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:35665
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:35665
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.87.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4371
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:49.721325 4371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:49.722226 4371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:49.729820 4383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:49.729879 4377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:49.729876 4378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:49.730010 4371 server_base.cc:1034] running on GCE node
I20260808 22:31:49.730540 4371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:49.731483 4371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:49.732636 4371 hybrid_clock.cc:648] HybridClock initialized: now 1786228309732623 us; error 24 us; skew 500 ppm
Aug 08 22:31:49 dist-test-slave-266d krb5kdc[4355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228309, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:31:49.739260 4371 init.cc:377] Logged in from keytab as kudu/127.31.87.190@KRBTEST.COM (short username kudu)
I20260808 22:31:49.740082 4371 webserver.cc:460] Webserver started at http://127.31.87.190:34711/ using document root <none> and password file <none>
I20260808 22:31:49.740576 4371 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:49.740639 4371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:49.740830 4371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:49.742734 4371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786228291013016-32094-0/minicluster-data/master-0/data/instance:
uuid: "9a5722af64fa4677af7a0e629e3d6278"
format_stamp: "Formatted at 2026-08-08 22:31:49 on dist-test-slave-266d"
server_key: "b58a1e575fb4b6bbd9b4fe52398e4fbc"
server_key_iv: "222beafffa4962cd55ca185c721038a7"
server_key_version: "encryptionkey@0"
I20260808 22:31:49.743254 4371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786228291013016-32094-0/minicluster-data/master-0/wal/instance:
uuid: "9a5722af64fa4677af7a0e629e3d6278"
format_stamp: "Formatted at 2026-08-08 22:31:49 on dist-test-slave-266d"
server_key: "b58a1e575fb4b6bbd9b4fe52398e4fbc"
server_key_iv: "222beafffa4962cd55ca185c721038a7"
server_key_version: "encryptionkey@0"
I20260808 22:31:49.746670 4371 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260808 22:31:49.748764 4395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:49.749867 4371 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 22:31:49.749974 4371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "9a5722af64fa4677af7a0e629e3d6278"
format_stamp: "Formatted at 2026-08-08 22:31:49 on dist-test-slave-266d"
server_key: "b58a1e575fb4b6bbd9b4fe52398e4fbc"
server_key_iv: "222beafffa4962cd55ca185c721038a7"
server_key_version: "encryptionkey@0"
I20260808 22:31:49.750082 4371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:49.772382 4371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:49.775012 4371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:49.775168 4371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:49.781489 4371 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.190:35665
I20260808 22:31:49.781803 4499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.190:35665 every 8 connection(s)
I20260808 22:31:49.782379 4371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:31:49.784941 4500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:49.790165 4500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a5722af64fa4677af7a0e629e3d6278: Bootstrap starting.
I20260808 22:31:49.790657 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 4371
I20260808 22:31:49.790757 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786228291013016-32094-0/minicluster-data/master-0/wal/instance
I20260808 22:31:49.790997 32094 external_mini_cluster.cc:1428] Setting key 9fa0347d759e9c91f39ed47813a46596
I20260808 22:31:49.792611 4500 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a5722af64fa4677af7a0e629e3d6278: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:49.793490 4500 log.cc:824] T 00000000000000000000000000000000 P 9a5722af64fa4677af7a0e629e3d6278: Log is configured to *not* fsync() on all Append() calls
I20260808 22:31:49.795836 4500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a5722af64fa4677af7a0e629e3d6278: No bootstrap required, opened a new log
Aug 08 22:31:49 dist-test-slave-266d krb5kdc[4355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:31:49.799904 4500 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a5722af64fa4677af7a0e629e3d6278 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a5722af64fa4677af7a0e629e3d6278" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 35665 } }
I20260808 22:31:49.800132 4500 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9a5722af64fa4677af7a0e629e3d6278 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:49.800189 4500 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a5722af64fa4677af7a0e629e3d6278 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a5722af64fa4677af7a0e629e3d6278, State: Initialized, Role: FOLLOWER
I20260808 22:31:49.800604 4500 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a5722af64fa4677af7a0e629e3d6278 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a5722af64fa4677af7a0e629e3d6278" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 35665 } }
I20260808 22:31:49.800730 4500 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a5722af64fa4677af7a0e629e3d6278 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:31:49.800802 4500 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a5722af64fa4677af7a0e629e3d6278 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:31:49.800896 4500 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a5722af64fa4677af7a0e629e3d6278 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:49.801919 4500 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a5722af64fa4677af7a0e629e3d6278 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a5722af64fa4677af7a0e629e3d6278" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 35665 } }
I20260808 22:31:49.802268 4500 leader_election.cc:302] T 00000000000000000000000000000000 P 9a5722af64fa4677af7a0e629e3d6278 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a5722af64fa4677af7a0e629e3d6278; no voters:
I20260808 22:31:49.802589 4500 leader_election.cc:288] T 00000000000000000000000000000000 P 9a5722af64fa4677af7a0e629e3d6278 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:31:49.802688 4509 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a5722af64fa4677af7a0e629e3d6278 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:31:49.802886 4509 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a5722af64fa4677af7a0e629e3d6278 [term 1 LEADER]: Becoming Leader. State: Replica: 9a5722af64fa4677af7a0e629e3d6278, State: Running, Role: LEADER
I20260808 22:31:49.803253 4509 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a5722af64fa4677af7a0e629e3d6278 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a5722af64fa4677af7a0e629e3d6278" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 35665 } }
I20260808 22:31:49.803725 4500 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a5722af64fa4677af7a0e629e3d6278 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:31:49.804863 4510 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a5722af64fa4677af7a0e629e3d6278 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9a5722af64fa4677af7a0e629e3d6278" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a5722af64fa4677af7a0e629e3d6278" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 35665 } } }
I20260808 22:31:49.804984 4510 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a5722af64fa4677af7a0e629e3d6278 [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:49.805253 4511 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a5722af64fa4677af7a0e629e3d6278 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a5722af64fa4677af7a0e629e3d6278. Latest consensus state: current_term: 1 leader_uuid: "9a5722af64fa4677af7a0e629e3d6278" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a5722af64fa4677af7a0e629e3d6278" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 35665 } } }
I20260808 22:31:49.805406 4523 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:31:49.805465 4511 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a5722af64fa4677af7a0e629e3d6278 [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:49.805696 4507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:49.792204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45678 (local address 127.31.87.190:35665)
0808 22:31:49.792614 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:49.792633 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:49.792676 (+ 43us) server_negotiation.cc:407] Connection header received
0808 22:31:49.793509 (+ 833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:49.793533 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:49.793936 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:49.794235 (+ 299us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:49.795078 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:49.796041 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:49.797216 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:49.797660 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:49.797772 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:49.800438 (+ 2666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:49.800443 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:49.800456 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:49.800482 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:49.802423 (+ 1941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:49.802925 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:49.802931 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:49.802937 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:49.803077 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:49.803426 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:49.803429 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:49.803431 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:49.803894 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:49.804131 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:49.804399 (+ 268us) server_negotiation.cc:299] Negotiation successful
0808 22:31:49.804698 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":108,"threads_started":1}
I20260808 22:31:49.808373 4523 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:31:49.809008 32094 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:31:49.813771 4523 catalog_manager.cc:1347] Generated new cluster ID: 2b3f0593dfff4623b15000a61efc13f3
I20260808 22:31:49.813827 4523 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:31:49.823302 4523 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:31:49.824276 4523 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:31:49.829131 4507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:49.822840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45686 (local address 127.31.87.190:35665)
0808 22:31:49.822961 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:49.822965 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:49.823829 (+ 864us) server_negotiation.cc:407] Connection header received
0808 22:31:49.823943 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:49.823946 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:49.824303 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:49.824374 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:49.825007 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:49.825812 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:49.826970 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:49.827301 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:49.827383 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:49.828204 (+ 821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:49.828208 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:49.828210 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:31:49.828676 (+ 466us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:49.828706 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:31:49.828723 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:49.828798 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:49.828916 (+ 118us) server_negotiation.cc:299] Negotiation successful
0808 22:31:49.828968 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":46}
I20260808 22:31:49.834532 4523 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a5722af64fa4677af7a0e629e3d6278: Generated new TSK 0
I20260808 22:31:49.835242 4523 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:31:49.844168 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 4371
2026-08-08T22:31:49Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1501 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:31:49 dist-test-slave-266d krb5kdc[4555](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:31:49 dist-test-slave-266d krb5kdc[4555](info): set up 2 sockets
Aug 08 22:31:49 dist-test-slave-266d krb5kdc[4555](info): commencing operation
krb5kdc: starting...
W20260808 22:31:50.991212 32094 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Aug 08 22:31:51 dist-test-slave-266d krb5kdc[4555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228311, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:31:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:31:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.87.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
I20260808 22:31:51.105713 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:45123
--webserver_interface=127.31.87.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:35267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:45123
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:31:51.212963 4574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:51.213204 4574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:51.213248 4574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:51.217142 4574 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:31:51.217188 4574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:51.217207 4574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:51.217237 4574 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:31:51.217262 4574 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:31:51.222234 4574 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:35267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:45123
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:45123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.87.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4574
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:51.223146 4574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:51.224051 4574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:51.231918 4587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:51.231918 4585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:51.231885 4584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:51.231957 4574 server_base.cc:1034] running on GCE node
I20260808 22:31:51.232671 4574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:51.233683 4574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:51.234839 4574 hybrid_clock.cc:648] HybridClock initialized: now 1786228311234828 us; error 31 us; skew 500 ppm
Aug 08 22:31:51 dist-test-slave-266d krb5kdc[4555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228311, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:31:51.237442 4574 init.cc:377] Logged in from keytab as kudu/127.31.87.190@KRBTEST.COM (short username kudu)
I20260808 22:31:51.238240 4574 webserver.cc:460] Webserver started at http://127.31.87.190:39195/ using document root <none> and password file <none>
I20260808 22:31:51.238729 4574 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:51.238783 4574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:51.238943 4574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:51.240859 4574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/master-0/data/instance:
uuid: "05f7435714fe44ce848cdb18ca79cdd9"
format_stamp: "Formatted at 2026-08-08 22:31:51 on dist-test-slave-266d"
server_key: "25fa8f8c3e825c70a7cabeb024c2a0c2"
server_key_iv: "ab4103e6a367be4c58fb3bc71ae5afa5"
server_key_version: "encryptionkey@0"
I20260808 22:31:51.241374 4574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/master-0/wal/instance:
uuid: "05f7435714fe44ce848cdb18ca79cdd9"
format_stamp: "Formatted at 2026-08-08 22:31:51 on dist-test-slave-266d"
server_key: "25fa8f8c3e825c70a7cabeb024c2a0c2"
server_key_iv: "ab4103e6a367be4c58fb3bc71ae5afa5"
server_key_version: "encryptionkey@0"
I20260808 22:31:51.244803 4574 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.001s
I20260808 22:31:51.247025 4594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:51.248152 4574 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:31:51.248262 4574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "05f7435714fe44ce848cdb18ca79cdd9"
format_stamp: "Formatted at 2026-08-08 22:31:51 on dist-test-slave-266d"
server_key: "25fa8f8c3e825c70a7cabeb024c2a0c2"
server_key_iv: "ab4103e6a367be4c58fb3bc71ae5afa5"
server_key_version: "encryptionkey@0"
I20260808 22:31:51.248350 4574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:51.258232 4574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:51.260640 4574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:51.260807 4574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:51.267167 4574 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.190:45123
I20260808 22:31:51.267452 4684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.190:45123 every 8 connection(s)
I20260808 22:31:51.268110 4574 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:31:51.270630 4685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:51.271116 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 4574
I20260808 22:31:51.271229 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/master-0/wal/instance
I20260808 22:31:51.271474 32094 external_mini_cluster.cc:1428] Setting key 0fd0a5a614a8765a8de0949a0ee88ae8
I20260808 22:31:51.277120 4685 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05f7435714fe44ce848cdb18ca79cdd9: Bootstrap starting.
I20260808 22:31:51.279580 4685 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 05f7435714fe44ce848cdb18ca79cdd9: Neither blocks nor log segments found. Creating new log.
Aug 08 22:31:51 dist-test-slave-266d krb5kdc[4555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228311, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:31:51.280247 4685 log.cc:824] T 00000000000000000000000000000000 P 05f7435714fe44ce848cdb18ca79cdd9: Log is configured to *not* fsync() on all Append() calls
I20260808 22:31:51.282419 4685 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05f7435714fe44ce848cdb18ca79cdd9: No bootstrap required, opened a new log
I20260808 22:31:51.285406 4689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:51.272767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60902 (local address 127.31.87.190:45123)
0808 22:31:51.273216 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:51.273232 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:51.273274 (+ 42us) server_negotiation.cc:407] Connection header received
0808 22:31:51.274063 (+ 789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:51.274086 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:51.274476 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:51.274823 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:51.275443 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:51.276909 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:51.277752 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:51.278423 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:51.278569 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:51.280814 (+ 2245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:51.280819 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:51.280831 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:51.280857 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:51.282616 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:51.282996 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:51.283002 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:51.283007 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:51.283140 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:51.283353 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:51.283356 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:51.283358 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:51.283821 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:51.284007 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:51.284192 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 22:31:51.284489 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":113,"threads_started":1}
I20260808 22:31:51.286677 4685 raft_consensus.cc:348] T 00000000000000000000000000000000 P 05f7435714fe44ce848cdb18ca79cdd9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05f7435714fe44ce848cdb18ca79cdd9" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45123 } }
I20260808 22:31:51.286953 4685 raft_consensus.cc:374] T 00000000000000000000000000000000 P 05f7435714fe44ce848cdb18ca79cdd9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:51.287036 4685 raft_consensus.cc:729] T 00000000000000000000000000000000 P 05f7435714fe44ce848cdb18ca79cdd9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05f7435714fe44ce848cdb18ca79cdd9, State: Initialized, Role: FOLLOWER
I20260808 22:31:51.287487 4685 consensus_queue.cc:260] T 00000000000000000000000000000000 P 05f7435714fe44ce848cdb18ca79cdd9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05f7435714fe44ce848cdb18ca79cdd9" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45123 } }
I20260808 22:31:51.287680 4685 raft_consensus.cc:388] T 00000000000000000000000000000000 P 05f7435714fe44ce848cdb18ca79cdd9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:31:51.287773 4685 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05f7435714fe44ce848cdb18ca79cdd9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:31:51.287881 4685 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 05f7435714fe44ce848cdb18ca79cdd9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:51.288966 4685 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05f7435714fe44ce848cdb18ca79cdd9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05f7435714fe44ce848cdb18ca79cdd9" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45123 } }
I20260808 22:31:51.289390 4685 leader_election.cc:302] T 00000000000000000000000000000000 P 05f7435714fe44ce848cdb18ca79cdd9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 05f7435714fe44ce848cdb18ca79cdd9; no voters:
I20260808 22:31:51.289765 4685 leader_election.cc:288] T 00000000000000000000000000000000 P 05f7435714fe44ce848cdb18ca79cdd9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:31:51.289875 4693 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 05f7435714fe44ce848cdb18ca79cdd9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:31:51.290109 4693 raft_consensus.cc:686] T 00000000000000000000000000000000 P 05f7435714fe44ce848cdb18ca79cdd9 [term 1 LEADER]: Becoming Leader. State: Replica: 05f7435714fe44ce848cdb18ca79cdd9, State: Running, Role: LEADER
I20260808 22:31:51.290427 4693 consensus_queue.cc:237] T 00000000000000000000000000000000 P 05f7435714fe44ce848cdb18ca79cdd9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05f7435714fe44ce848cdb18ca79cdd9" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45123 } }
I20260808 22:31:51.291419 4685 sys_catalog.cc:564] T 00000000000000000000000000000000 P 05f7435714fe44ce848cdb18ca79cdd9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:31:51.292115 4698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05f7435714fe44ce848cdb18ca79cdd9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "05f7435714fe44ce848cdb18ca79cdd9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05f7435714fe44ce848cdb18ca79cdd9" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45123 } } }
I20260808 22:31:51.292281 4698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05f7435714fe44ce848cdb18ca79cdd9 [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:51.292156 4696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05f7435714fe44ce848cdb18ca79cdd9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 05f7435714fe44ce848cdb18ca79cdd9. Latest consensus state: current_term: 1 leader_uuid: "05f7435714fe44ce848cdb18ca79cdd9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05f7435714fe44ce848cdb18ca79cdd9" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45123 } } }
I20260808 22:31:51.292620 4696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05f7435714fe44ce848cdb18ca79cdd9 [sys.catalog]: This master's current role is: LEADER
W20260808 22:31:51.299335 4720 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 05f7435714fe44ce848cdb18ca79cdd9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:31:51.299401 4720 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:31:51.299515 4705 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:31:51.300431 4705 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:31:51.305833 4705 catalog_manager.cc:1347] Generated new cluster ID: bad0b1df8b764c6cbaa68380d215219c
I20260808 22:31:51.305883 4705 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:31:51.335850 4705 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:31:51.336781 4705 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:31:51.343257 4705 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 05f7435714fe44ce848cdb18ca79cdd9: Generated new TSK 0
I20260808 22:31:51.343873 4705 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.87.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab.
Entry for principal kudu/127.31.87.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab.
I20260808 22:31:51.377871 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.129:0
--local_ip_for_outbound_sockets=127.31.87.129
--webserver_interface=127.31.87.129
--webserver_port=0
--tserver_master_addrs=127.31.87.190:45123
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab
--principal=kudu/127.31.87.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:35267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:31:51.485117 4730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:51.485365 4730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:51.485406 4730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:51.489283 4730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:51.489332 4730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:51.489399 4730 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.129
I20260808 22:31:51.494526 4730 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:35267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab
--principal=kudu/127.31.87.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.87.129
--webserver_port=0
--tserver_master_addrs=127.31.87.190:45123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4730
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.129
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:51.495473 4730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:51.496451 4730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:51.505713 4736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:51.505704 4737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:51.505776 4730 server_base.cc:1034] running on GCE node
W20260808 22:31:51.505705 4744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:51.506266 4730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:51.506829 4730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:51.507987 4730 hybrid_clock.cc:648] HybridClock initialized: now 1786228311507971 us; error 27 us; skew 500 ppm
Aug 08 22:31:51 dist-test-slave-266d krb5kdc[4555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228311, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:31:51.510501 4730 init.cc:377] Logged in from keytab as kudu/127.31.87.129@KRBTEST.COM (short username kudu)
I20260808 22:31:51.511287 4730 webserver.cc:460] Webserver started at http://127.31.87.129:38425/ using document root <none> and password file <none>
I20260808 22:31:51.511799 4730 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:51.511859 4730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:51.512070 4730 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:51.514045 4730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-0/data/instance:
uuid: "4bc043f2611f4efeab77c4b42fe81ced"
format_stamp: "Formatted at 2026-08-08 22:31:51 on dist-test-slave-266d"
server_key: "39087f043101a9ac23a2a5b6f382f699"
server_key_iv: "09e834230136ac3970c6ab41b126fcba"
server_key_version: "encryptionkey@0"
I20260808 22:31:51.514551 4730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-0/wal/instance:
uuid: "4bc043f2611f4efeab77c4b42fe81ced"
format_stamp: "Formatted at 2026-08-08 22:31:51 on dist-test-slave-266d"
server_key: "39087f043101a9ac23a2a5b6f382f699"
server_key_iv: "09e834230136ac3970c6ab41b126fcba"
server_key_version: "encryptionkey@0"
I20260808 22:31:51.518006 4730 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:31:51.520179 4752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:51.521370 4730 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:31:51.521482 4730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-0/wal
uuid: "4bc043f2611f4efeab77c4b42fe81ced"
format_stamp: "Formatted at 2026-08-08 22:31:51 on dist-test-slave-266d"
server_key: "39087f043101a9ac23a2a5b6f382f699"
server_key_iv: "09e834230136ac3970c6ab41b126fcba"
server_key_version: "encryptionkey@0"
I20260808 22:31:51.521569 4730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:51.541137 4730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:51.543771 4730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:51.543941 4730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:51.544466 4730 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:31:51.545266 4730 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:31:51.545316 4730 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:51.545369 4730 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:31:51.545395 4730 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:51.553596 4730 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.129:43891
I20260808 22:31:51.553622 4926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.129:43891 every 8 connection(s)
I20260808 22:31:51.554405 4730 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
Aug 08 22:31:51 dist-test-slave-266d krb5kdc[4555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228311, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.129@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:31:51.563609 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 4730
I20260808 22:31:51.563738 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-0/wal/instance
I20260808 22:31:51.564043 32094 external_mini_cluster.cc:1428] Setting key 1322552e1b2b838609888f9cd9a8dcb3
I20260808 22:31:51.566828 4689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:51.556049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:53027 (local address 127.31.87.190:45123)
0808 22:31:51.556182 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:51.556186 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:51.556981 (+ 795us) server_negotiation.cc:407] Connection header received
0808 22:31:51.558290 (+ 1309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:51.558293 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:51.558340 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:51.558412 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:51.559456 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:51.559971 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:51.561000 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:51.561259 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:51.561334 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:51.563412 (+ 2078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:51.563415 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:51.563417 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:51.563441 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:51.564851 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:51.565436 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:51.565440 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:51.565442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:51.565522 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:51.565827 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:51.565831 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:51.565832 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:51.566013 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:51.566101 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:51.566547 (+ 446us) server_negotiation.cc:299] Negotiation successful
0808 22:31:51.566667 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 22:31:51.567528 4931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:51.556413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45123 (local address 127.31.87.129:53027)
0808 22:31:51.556832 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:51.556872 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:51.558101 (+ 1229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:51.558552 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:51.558559 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:51.558892 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:51.559340 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:51.559353 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:51.560057 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:51.560059 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:51.560912 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:51.560920 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:51.561365 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:51.561368 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:51.561498 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:51.561943 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:31:51.561957 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:51.563259 (+ 1302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:51.564958 (+ 1699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:51.564964 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:51.564977 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:51.565343 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:51.565605 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:51.565609 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:51.565611 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:51.565737 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:51.566110 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:51.566116 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:51.566329 (+ 213us) client_negotiation.cc:769] Sending connection context
0808 22:31:51.566487 (+ 158us) client_negotiation.cc:240] Negotiation successful
0808 22:31:51.566772 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":114,"threads_started":1}
I20260808 22:31:51.568730 4928 heartbeater.cc:347] Connected to a master server at 127.31.87.190:45123
I20260808 22:31:51.568940 4928 heartbeater.cc:464] Registering TS with master...
I20260808 22:31:51.569407 4928 heartbeater.cc:511] Master 127.31.87.190:45123 requested a full tablet report, sending...
I20260808 22:31:51.570925 4624 ts_manager.cc:194] Registered new tserver with Master: 4bc043f2611f4efeab77c4b42fe81ced (127.31.87.129:43891)
I20260808 22:31:51.572438 4624 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.129@KRBTEST.COM'} at 127.31.87.129:53027
WARNING: no policy specified for kudu/127.31.87.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab.
Entry for principal kudu/127.31.87.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab.
I20260808 22:31:51.591405 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.130:0
--local_ip_for_outbound_sockets=127.31.87.130
--webserver_interface=127.31.87.130
--webserver_port=0
--tserver_master_addrs=127.31.87.190:45123
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab
--principal=kudu/127.31.87.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:35267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:31:51.700570 4934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:51.700801 4934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:51.700841 4934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:51.704882 4934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:51.704936 4934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:51.704991 4934 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.130
I20260808 22:31:51.710005 4934 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:35267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab
--principal=kudu/127.31.87.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.87.130
--webserver_port=0
--tserver_master_addrs=127.31.87.190:45123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4934
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.130
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:51.710881 4934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:51.711786 4934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:51.720813 4943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:51.720846 4945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:51.720866 4934 server_base.cc:1034] running on GCE node
W20260808 22:31:51.720824 4941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:51.721385 4934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:51.721932 4934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:51.723083 4934 hybrid_clock.cc:648] HybridClock initialized: now 1786228311723058 us; error 34 us; skew 500 ppm
Aug 08 22:31:51 dist-test-slave-266d krb5kdc[4555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228311, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:31:51.725638 4934 init.cc:377] Logged in from keytab as kudu/127.31.87.130@KRBTEST.COM (short username kudu)
I20260808 22:31:51.726414 4934 webserver.cc:460] Webserver started at http://127.31.87.130:41969/ using document root <none> and password file <none>
I20260808 22:31:51.726933 4934 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:51.726996 4934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:51.727186 4934 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:51.729041 4934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-1/data/instance:
uuid: "a708d35c28194cbeae345a05fd9a7171"
format_stamp: "Formatted at 2026-08-08 22:31:51 on dist-test-slave-266d"
server_key: "92f1f1d0f7e4c956c16c2fbd409fec15"
server_key_iv: "666f5279efcaa604d42900603c8a90b1"
server_key_version: "encryptionkey@0"
I20260808 22:31:51.729599 4934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-1/wal/instance:
uuid: "a708d35c28194cbeae345a05fd9a7171"
format_stamp: "Formatted at 2026-08-08 22:31:51 on dist-test-slave-266d"
server_key: "92f1f1d0f7e4c956c16c2fbd409fec15"
server_key_iv: "666f5279efcaa604d42900603c8a90b1"
server_key_version: "encryptionkey@0"
I20260808 22:31:51.732942 4934 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 22:31:51.735109 4956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:51.736253 4934 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:31:51.736375 4934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-1/wal
uuid: "a708d35c28194cbeae345a05fd9a7171"
format_stamp: "Formatted at 2026-08-08 22:31:51 on dist-test-slave-266d"
server_key: "92f1f1d0f7e4c956c16c2fbd409fec15"
server_key_iv: "666f5279efcaa604d42900603c8a90b1"
server_key_version: "encryptionkey@0"
I20260808 22:31:51.736465 4934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:51.759716 4934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:51.762247 4934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:51.762411 4934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:51.762951 4934 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:31:51.763715 4934 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:31:51.763761 4934 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:51.763811 4934 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:31:51.763840 4934 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:51.771914 4934 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.130:35339
I20260808 22:31:51.771940 5144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.130:35339 every 8 connection(s)
I20260808 22:31:51.772724 4934 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
I20260808 22:31:51.777359 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 4934
I20260808 22:31:51.777483 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-1/wal/instance
I20260808 22:31:51.777777 32094 external_mini_cluster.cc:1428] Setting key b8dbdbfaddcee37ceb4605976ab5c63f
Aug 08 22:31:51 dist-test-slave-266d krb5kdc[4555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228311, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.130@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:31:51.786119 4689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:51.774339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:52909 (local address 127.31.87.190:45123)
0808 22:31:51.774450 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:51.774454 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:51.775187 (+ 733us) server_negotiation.cc:407] Connection header received
0808 22:31:51.776557 (+ 1370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:51.776561 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:51.776623 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:51.776694 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:51.778085 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:51.778760 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:51.780129 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:51.780350 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:51.780429 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:51.782845 (+ 2416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:51.782848 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:51.782850 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:51.782878 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:51.784170 (+ 1292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:51.784627 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:51.784631 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:51.784633 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:51.784721 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:51.785053 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:51.785056 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:51.785058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:51.785302 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:51.785408 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:51.785843 (+ 435us) server_negotiation.cc:299] Negotiation successful
0808 22:31:51.785960 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 22:31:51.786782 5148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:51.774641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45123 (local address 127.31.87.130:52909)
0808 22:31:51.775053 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:51.775091 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:51.776357 (+ 1266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:51.776863 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:51.776869 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:51.777299 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:51.777923 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:51.777944 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:51.778876 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:51.778879 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:51.780008 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:51.780020 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:51.780482 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:51.780486 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:51.780658 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:51.781307 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:31:51.781326 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:51.782695 (+ 1369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:51.784287 (+ 1592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:51.784292 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:51.784302 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:51.784530 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:51.784829 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:51.784832 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:51.784833 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:51.784965 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:51.785395 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:51.785400 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:51.785610 (+ 210us) client_negotiation.cc:769] Sending connection context
0808 22:31:51.785765 (+ 155us) client_negotiation.cc:240] Negotiation successful
0808 22:31:51.785992 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":119,"threads_started":1}
I20260808 22:31:51.787994 5145 heartbeater.cc:347] Connected to a master server at 127.31.87.190:45123
WARNING: no policy specified for kudu/127.31.87.131@KRBTEST.COM; defaulting to no policy
I20260808 22:31:51.788209 5145 heartbeater.cc:464] Registering TS with master...
I20260808 22:31:51.788635 5145 heartbeater.cc:511] Master 127.31.87.190:45123 requested a full tablet report, sending...
I20260808 22:31:51.789618 4624 ts_manager.cc:194] Registered new tserver with Master: a708d35c28194cbeae345a05fd9a7171 (127.31.87.130:35339)
I20260808 22:31:51.790148 4624 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.130@KRBTEST.COM'} at 127.31.87.130:52909
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab.
Entry for principal kudu/127.31.87.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab.
I20260808 22:31:51.804548 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.131:0
--local_ip_for_outbound_sockets=127.31.87.131
--webserver_interface=127.31.87.131
--webserver_port=0
--tserver_master_addrs=127.31.87.190:45123
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab
--principal=kudu/127.31.87.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:35267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:31:51.914027 5151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:51.914247 5151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:51.914285 5151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:51.918269 5151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:51.918320 5151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:51.918373 5151 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.131
I20260808 22:31:51.923322 5151 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:35267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab
--principal=kudu/127.31.87.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.87.131
--webserver_port=0
--tserver_master_addrs=127.31.87.190:45123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5151
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.131
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:51.924237 5151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:51.925213 5151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:51.934253 5165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:51.934240 5161 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:51.934391 5151 server_base.cc:1034] running on GCE node
W20260808 22:31:51.934298 5163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:51.934892 5151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:51.935503 5151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:51.936676 5151 hybrid_clock.cc:648] HybridClock initialized: now 1786228311936658 us; error 28 us; skew 500 ppm
Aug 08 22:31:51 dist-test-slave-266d krb5kdc[4555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228311, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:31:51.939378 5151 init.cc:377] Logged in from keytab as kudu/127.31.87.131@KRBTEST.COM (short username kudu)
I20260808 22:31:51.940308 5151 webserver.cc:460] Webserver started at http://127.31.87.131:41179/ using document root <none> and password file <none>
I20260808 22:31:51.940836 5151 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:51.940904 5151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:51.941164 5151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:51.943131 5151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-2/data/instance:
uuid: "579e5684383045559e05177e940bddec"
format_stamp: "Formatted at 2026-08-08 22:31:51 on dist-test-slave-266d"
server_key: "d51239262d0895b9fa1f2bcd55faf47f"
server_key_iv: "f5be1e7b00b19c10ad1ef11f409af0ca"
server_key_version: "encryptionkey@0"
I20260808 22:31:51.943714 5151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-2/wal/instance:
uuid: "579e5684383045559e05177e940bddec"
format_stamp: "Formatted at 2026-08-08 22:31:51 on dist-test-slave-266d"
server_key: "d51239262d0895b9fa1f2bcd55faf47f"
server_key_iv: "f5be1e7b00b19c10ad1ef11f409af0ca"
server_key_version: "encryptionkey@0"
I20260808 22:31:51.947610 5151 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260808 22:31:51.949956 5176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:51.951141 5151 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.002s
I20260808 22:31:51.951247 5151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-2/wal
uuid: "579e5684383045559e05177e940bddec"
format_stamp: "Formatted at 2026-08-08 22:31:51 on dist-test-slave-266d"
server_key: "d51239262d0895b9fa1f2bcd55faf47f"
server_key_iv: "f5be1e7b00b19c10ad1ef11f409af0ca"
server_key_version: "encryptionkey@0"
I20260808 22:31:51.951345 5151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:51.963394 5151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:51.966212 5151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:51.966395 5151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:51.966965 5151 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:31:51.967805 5151 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:31:51.967859 5151 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:51.967916 5151 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:31:51.967947 5151 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:51.977139 5151 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.131:33171
I20260808 22:31:51.977176 5376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.131:33171 every 8 connection(s)
I20260808 22:31:51.978012 5151 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
I20260808 22:31:51.980476 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 5151
I20260808 22:31:51.980618 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786228291013016-32094-0/minicluster-data/ts-2/wal/instance
I20260808 22:31:51.980976 32094 external_mini_cluster.cc:1428] Setting key ff38130c0722bf93d03501e77fd0de55
Aug 08 22:31:51 dist-test-slave-266d krb5kdc[4555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228311, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.131@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:31:51.991449 4689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:51.979698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:49241 (local address 127.31.87.190:45123)
0808 22:31:51.979824 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:51.979828 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:51.980666 (+ 838us) server_negotiation.cc:407] Connection header received
0808 22:31:51.982430 (+ 1764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:51.982434 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:51.982499 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:51.982578 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:51.983804 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:51.984338 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:51.985550 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:51.985769 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:51.985862 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:51.988258 (+ 2396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:51.988262 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:51.988264 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:51.988289 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:51.989553 (+ 1264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:51.990034 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:51.990038 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:51.990039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:51.990125 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:51.990431 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:51.990434 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:51.990436 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:51.990607 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:51.990694 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:51.991164 (+ 470us) server_negotiation.cc:299] Negotiation successful
0808 22:31:51.991288 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 22:31:51.992271 5381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:51.979990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45123 (local address 127.31.87.131:49241)
0808 22:31:51.980478 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:51.980536 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:51.982191 (+ 1655us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:51.982770 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:51.982777 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:51.983163 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:51.983660 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:51.983675 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:51.984468 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:51.984472 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:51.985431 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:51.985443 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:51.985924 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:51.985928 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:51.986092 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:51.986630 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:31:51.986648 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:51.988095 (+ 1447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:51.989679 (+ 1584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:51.989685 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:51.989700 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:51.989941 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:51.990210 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:51.990213 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:51.990215 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:51.990344 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:51.990708 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:51.990715 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:51.990938 (+ 223us) client_negotiation.cc:769] Sending connection context
0808 22:31:51.991097 (+ 159us) client_negotiation.cc:240] Negotiation successful
0808 22:31:51.991405 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":113,"threads_started":1}
I20260808 22:31:51.993324 5377 heartbeater.cc:347] Connected to a master server at 127.31.87.190:45123
I20260808 22:31:51.993549 5377 heartbeater.cc:464] Registering TS with master...
I20260808 22:31:51.994055 5377 heartbeater.cc:511] Master 127.31.87.190:45123 requested a full tablet report, sending...
I20260808 22:31:51.995060 4624 ts_manager.cc:194] Registered new tserver with Master: 579e5684383045559e05177e940bddec (127.31.87.131:33171)
I20260808 22:31:51.995613 4624 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.131@KRBTEST.COM'} at 127.31.87.131:49241
I20260808 22:31:52.006317 32094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 22:31:52 dist-test-slave-266d krb5kdc[4555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228312, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 08 22:31:52 dist-test-slave-266d krb5kdc[4555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228312, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:31:52.031498 4689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:52.023818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60912 (local address 127.31.87.190:45123)
0808 22:31:52.023933 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:52.023937 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:52.024206 (+ 269us) server_negotiation.cc:407] Connection header received
0808 22:31:52.024353 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:52.024356 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:52.024397 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:52.024456 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:52.025138 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:52.025667 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:52.026488 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:52.026695 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:52.026766 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:52.028891 (+ 2125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:52.028894 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:52.028896 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:52.028921 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:52.030198 (+ 1277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:52.030548 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:52.030551 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:52.030552 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:52.030617 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:52.030803 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:52.030805 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:52.030806 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:52.030996 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:52.031091 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:52.031243 (+ 152us) server_negotiation.cc:299] Negotiation successful
0808 22:31:52.031344 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260808 22:31:52.031836 4624 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:60912
I20260808 22:31:52.032550 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 4730
I20260808 22:31:52.061378 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 4934
I20260808 22:31:52.068337 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 5151
I20260808 22:31:52.077675 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 4574
2026-08-08T22:31:52Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2224 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:31:52 dist-test-slave-266d krb5kdc[5399](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:31:52 dist-test-slave-266d krb5kdc[5399](info): set up 2 sockets
Aug 08 22:31:52 dist-test-slave-266d krb5kdc[5399](info): commencing operation
krb5kdc: starting...
W20260808 22:31:53.237463 32094 mini_kdc.cc:121] Time spent starting KDC: real 1.135s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Aug 08 22:31:53 dist-test-slave-266d krb5kdc[5399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228313, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:31:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:31:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.87.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
I20260808 22:31:53.349351 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:38219
--webserver_interface=127.31.87.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:42297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:38219
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:31:53.454610 5419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:53.454856 5419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:53.454908 5419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:53.458833 5419 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:31:53.458890 5419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:53.458916 5419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:53.458936 5419 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:31:53.458957 5419 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:31:53.464040 5419 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:42297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:38219
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:38219
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.87.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:53.464985 5419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:53.465952 5419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:53.473989 5427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:53.473984 5426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:53.473981 5431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:53.474068 5419 server_base.cc:1034] running on GCE node
I20260808 22:31:53.474771 5419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:53.475762 5419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:53.476931 5419 hybrid_clock.cc:648] HybridClock initialized: now 1786228313476920 us; error 28 us; skew 500 ppm
Aug 08 22:31:53 dist-test-slave-266d krb5kdc[5399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228313, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:31:53.479561 5419 init.cc:377] Logged in from keytab as kudu/127.31.87.190@KRBTEST.COM (short username kudu)
I20260808 22:31:53.480387 5419 webserver.cc:460] Webserver started at http://127.31.87.190:38221/ using document root <none> and password file <none>
I20260808 22:31:53.480866 5419 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:53.480921 5419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:53.481082 5419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:53.482997 5419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/master-0/data/instance:
uuid: "7ba83954a16141c5ac316faed064877c"
format_stamp: "Formatted at 2026-08-08 22:31:53 on dist-test-slave-266d"
server_key: "cdcdfa162809e9cea34c9008c9322451"
server_key_iv: "4b80e0c0afab9b294e2ae22101d7a5b0"
server_key_version: "encryptionkey@0"
I20260808 22:31:53.483501 5419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/master-0/wal/instance:
uuid: "7ba83954a16141c5ac316faed064877c"
format_stamp: "Formatted at 2026-08-08 22:31:53 on dist-test-slave-266d"
server_key: "cdcdfa162809e9cea34c9008c9322451"
server_key_iv: "4b80e0c0afab9b294e2ae22101d7a5b0"
server_key_version: "encryptionkey@0"
I20260808 22:31:53.486910 5419 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 22:31:53.489048 5446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:53.490186 5419 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:31:53.490281 5419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "7ba83954a16141c5ac316faed064877c"
format_stamp: "Formatted at 2026-08-08 22:31:53 on dist-test-slave-266d"
server_key: "cdcdfa162809e9cea34c9008c9322451"
server_key_iv: "4b80e0c0afab9b294e2ae22101d7a5b0"
server_key_version: "encryptionkey@0"
I20260808 22:31:53.490360 5419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:53.521173 5419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:53.524240 5419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:53.524397 5419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:53.531524 5419 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.190:38219
I20260808 22:31:53.531534 5532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.190:38219 every 8 connection(s)
I20260808 22:31:53.532496 5419 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:31:53.534372 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 5419
I20260808 22:31:53.534487 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/master-0/wal/instance
I20260808 22:31:53.534771 32094 external_mini_cluster.cc:1428] Setting key e7e7d03c0223c3e48966ba22e3180e7b
I20260808 22:31:53.535301 5534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:53.542017 5534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ba83954a16141c5ac316faed064877c: Bootstrap starting.
Aug 08 22:31:53 dist-test-slave-266d krb5kdc[5399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228313, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:31:53.544162 5534 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ba83954a16141c5ac316faed064877c: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:53.544939 5534 log.cc:824] T 00000000000000000000000000000000 P 7ba83954a16141c5ac316faed064877c: Log is configured to *not* fsync() on all Append() calls
I20260808 22:31:53.546761 5534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ba83954a16141c5ac316faed064877c: No bootstrap required, opened a new log
I20260808 22:31:53.549119 5539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:53.536163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40252 (local address 127.31.87.190:38219)
0808 22:31:53.536607 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:53.536625 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:53.536666 (+ 41us) server_negotiation.cc:407] Connection header received
0808 22:31:53.537454 (+ 788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:53.537478 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:53.537883 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:53.538253 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:53.538949 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:53.540367 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:53.541184 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:53.541835 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:53.541984 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:53.544448 (+ 2464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:53.544453 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:53.544466 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:53.544494 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:53.546144 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:53.546545 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:53.546550 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:53.546554 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:53.546670 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:53.546926 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:53.546930 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:53.546932 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:53.547449 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:53.547665 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:53.547845 (+ 180us) server_negotiation.cc:299] Negotiation successful
0808 22:31:53.548142 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":144,"threads_started":1}
I20260808 22:31:53.550752 5534 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ba83954a16141c5ac316faed064877c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ba83954a16141c5ac316faed064877c" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 38219 } }
I20260808 22:31:53.550966 5534 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ba83954a16141c5ac316faed064877c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:53.551047 5534 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ba83954a16141c5ac316faed064877c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ba83954a16141c5ac316faed064877c, State: Initialized, Role: FOLLOWER
I20260808 22:31:53.551460 5534 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ba83954a16141c5ac316faed064877c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ba83954a16141c5ac316faed064877c" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 38219 } }
I20260808 22:31:53.551606 5534 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7ba83954a16141c5ac316faed064877c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:31:53.551677 5534 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ba83954a16141c5ac316faed064877c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:31:53.551779 5534 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ba83954a16141c5ac316faed064877c [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:53.552608 5534 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ba83954a16141c5ac316faed064877c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ba83954a16141c5ac316faed064877c" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 38219 } }
I20260808 22:31:53.552917 5534 leader_election.cc:302] T 00000000000000000000000000000000 P 7ba83954a16141c5ac316faed064877c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7ba83954a16141c5ac316faed064877c; no voters:
I20260808 22:31:53.553246 5534 leader_election.cc:288] T 00000000000000000000000000000000 P 7ba83954a16141c5ac316faed064877c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:31:53.553373 5541 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ba83954a16141c5ac316faed064877c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:31:53.553609 5541 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ba83954a16141c5ac316faed064877c [term 1 LEADER]: Becoming Leader. State: Replica: 7ba83954a16141c5ac316faed064877c, State: Running, Role: LEADER
I20260808 22:31:53.553920 5541 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ba83954a16141c5ac316faed064877c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ba83954a16141c5ac316faed064877c" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 38219 } }
I20260808 22:31:53.554425 5534 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ba83954a16141c5ac316faed064877c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:31:53.554960 5547 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ba83954a16141c5ac316faed064877c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ba83954a16141c5ac316faed064877c. Latest consensus state: current_term: 1 leader_uuid: "7ba83954a16141c5ac316faed064877c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ba83954a16141c5ac316faed064877c" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 38219 } } }
I20260808 22:31:53.555178 5547 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ba83954a16141c5ac316faed064877c [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:53.555364 5542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ba83954a16141c5ac316faed064877c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7ba83954a16141c5ac316faed064877c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ba83954a16141c5ac316faed064877c" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 38219 } } }
I20260808 22:31:53.555447 5542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ba83954a16141c5ac316faed064877c [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:53.555665 5557 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:31:53.558597 5557 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:31:53.563714 5557 catalog_manager.cc:1347] Generated new cluster ID: 5e9651fab1ac427f92cbb190fd7a3e94
I20260808 22:31:53.563774 5557 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:31:53.591637 5557 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:31:53.592520 5557 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:31:53.600291 5557 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7ba83954a16141c5ac316faed064877c: Generated new TSK 0
I20260808 22:31:53.600934 5557 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.87.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab.
Entry for principal kudu/127.31.87.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab.
I20260808 22:31:53.630925 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.129:0
--local_ip_for_outbound_sockets=127.31.87.129
--webserver_interface=127.31.87.129
--webserver_port=0
--tserver_master_addrs=127.31.87.190:38219
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab
--principal=kudu/127.31.87.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:42297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:31:53.737303 5577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:53.737511 5577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:53.737548 5577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:53.741403 5577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:53.741451 5577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:53.741509 5577 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.129
I20260808 22:31:53.746495 5577 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:42297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab
--principal=kudu/127.31.87.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.87.129
--webserver_port=0
--tserver_master_addrs=127.31.87.190:38219
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5577
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.129
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:53.747398 5577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:53.748313 5577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:53.757452 5587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:53.757496 5592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:53.757449 5588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:53.757715 5577 server_base.cc:1034] running on GCE node
I20260808 22:31:53.758136 5577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:53.758719 5577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:53.759878 5577 hybrid_clock.cc:648] HybridClock initialized: now 1786228313759852 us; error 43 us; skew 500 ppm
Aug 08 22:31:53 dist-test-slave-266d krb5kdc[5399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228313, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:31:53.762434 5577 init.cc:377] Logged in from keytab as kudu/127.31.87.129@KRBTEST.COM (short username kudu)
I20260808 22:31:53.763226 5577 webserver.cc:460] Webserver started at http://127.31.87.129:34061/ using document root <none> and password file <none>
I20260808 22:31:53.763738 5577 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:53.763803 5577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:53.763990 5577 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:53.765950 5577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/ts-0/data/instance:
uuid: "07dd2b98f6b14372bafeba1b6a591a8c"
format_stamp: "Formatted at 2026-08-08 22:31:53 on dist-test-slave-266d"
server_key: "37394ce2af252106d76b2710568f367c"
server_key_iv: "7ab129bd03d46827ff9e118896067be4"
server_key_version: "encryptionkey@0"
I20260808 22:31:53.766460 5577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/ts-0/wal/instance:
uuid: "07dd2b98f6b14372bafeba1b6a591a8c"
format_stamp: "Formatted at 2026-08-08 22:31:53 on dist-test-slave-266d"
server_key: "37394ce2af252106d76b2710568f367c"
server_key_iv: "7ab129bd03d46827ff9e118896067be4"
server_key_version: "encryptionkey@0"
I20260808 22:31:53.769881 5577 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 22:31:53.772015 5610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:53.773131 5577 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:31:53.773234 5577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/ts-0/wal
uuid: "07dd2b98f6b14372bafeba1b6a591a8c"
format_stamp: "Formatted at 2026-08-08 22:31:53 on dist-test-slave-266d"
server_key: "37394ce2af252106d76b2710568f367c"
server_key_iv: "7ab129bd03d46827ff9e118896067be4"
server_key_version: "encryptionkey@0"
I20260808 22:31:53.773317 5577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:53.807199 5577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:53.810220 5577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:53.810395 5577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:53.810966 5577 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:31:53.811797 5577 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:31:53.811846 5577 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:53.811899 5577 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:31:53.811928 5577 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:53.820132 5577 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.129:40393
I20260808 22:31:53.820165 5794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.129:40393 every 8 connection(s)
I20260808 22:31:53.820984 5577 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
I20260808 22:31:53.826586 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 5577
I20260808 22:31:53.826715 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/ts-0/wal/instance
I20260808 22:31:53.826975 32094 external_mini_cluster.cc:1428] Setting key 1d1366c8850f0b2cfd410d3a7ca51c56
Aug 08 22:31:53 dist-test-slave-266d krb5kdc[5399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228313, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.129@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:31:53.834303 5539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:53.822713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:52583 (local address 127.31.87.190:38219)
0808 22:31:53.822857 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:53.822861 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:53.823637 (+ 776us) server_negotiation.cc:407] Connection header received
0808 22:31:53.824960 (+ 1323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:53.824964 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:53.825024 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:53.825122 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:53.826244 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:53.826846 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:53.828072 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:53.828382 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:53.828480 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:53.830981 (+ 2501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:53.830986 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:53.830989 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:53.831019 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:53.832379 (+ 1360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:53.832835 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:53.832838 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:53.832840 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:53.832931 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:53.833279 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:53.833282 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:53.833284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:53.833512 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:53.833614 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:53.834069 (+ 455us) server_negotiation.cc:299] Negotiation successful
0808 22:31:53.834184 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 22:31:53.835053 5797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:53.823059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:38219 (local address 127.31.87.129:52583)
0808 22:31:53.823478 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:53.823519 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:53.824757 (+ 1238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:53.825299 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:53.825306 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:53.825643 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:53.826090 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:53.826113 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:53.826960 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:53.826963 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:53.827948 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:53.827959 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:53.828529 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:53.828533 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:53.828699 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:53.829291 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:31:53.829308 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:53.830815 (+ 1507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:53.832485 (+ 1670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:53.832489 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:53.832501 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:53.832743 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:53.833011 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:53.833013 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:53.833014 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:53.833191 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:53.833609 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:53.833614 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:53.833838 (+ 224us) client_negotiation.cc:769] Sending connection context
0808 22:31:53.833985 (+ 147us) client_negotiation.cc:240] Negotiation successful
0808 22:31:53.834245 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":115,"threads_started":1}
I20260808 22:31:53.836225 5795 heartbeater.cc:347] Connected to a master server at 127.31.87.190:38219
I20260808 22:31:53.836501 5795 heartbeater.cc:464] Registering TS with master...
I20260808 22:31:53.837071 5795 heartbeater.cc:511] Master 127.31.87.190:38219 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.31.87.130@KRBTEST.COM; defaulting to no policy
I20260808 22:31:53.838935 5474 ts_manager.cc:194] Registered new tserver with Master: 07dd2b98f6b14372bafeba1b6a591a8c (127.31.87.129:40393)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.130@KRBTEST.COM" created.
I20260808 22:31:53.840615 5474 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.129@KRBTEST.COM'} at 127.31.87.129:52583
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab.
Entry for principal kudu/127.31.87.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab.
I20260808 22:31:53.856076 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.130:0
--local_ip_for_outbound_sockets=127.31.87.130
--webserver_interface=127.31.87.130
--webserver_port=0
--tserver_master_addrs=127.31.87.190:38219
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab
--principal=kudu/127.31.87.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:42297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:31:53.962728 5802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:53.962960 5802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:53.963024 5802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:53.966959 5802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:53.967012 5802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:53.967087 5802 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.130
I20260808 22:31:53.972030 5802 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:42297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab
--principal=kudu/127.31.87.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.87.130
--webserver_port=0
--tserver_master_addrs=127.31.87.190:38219
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.130
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:53.972955 5802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:53.973919 5802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:53.982894 5816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:53.982915 5811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:53.982875 5814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:53.983178 5802 server_base.cc:1034] running on GCE node
I20260808 22:31:53.983541 5802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:53.984093 5802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:53.985248 5802 hybrid_clock.cc:648] HybridClock initialized: now 1786228313985229 us; error 25 us; skew 500 ppm
Aug 08 22:31:53 dist-test-slave-266d krb5kdc[5399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228313, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:31:53.987670 5802 init.cc:377] Logged in from keytab as kudu/127.31.87.130@KRBTEST.COM (short username kudu)
I20260808 22:31:53.988472 5802 webserver.cc:460] Webserver started at http://127.31.87.130:38951/ using document root <none> and password file <none>
I20260808 22:31:53.988965 5802 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:53.989028 5802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:53.989254 5802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:53.991114 5802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/ts-1/data/instance:
uuid: "e54274ae7ce2448fb343e0868f28fb06"
format_stamp: "Formatted at 2026-08-08 22:31:53 on dist-test-slave-266d"
server_key: "1d69033fcc2fac9bf93856495b01d4cb"
server_key_iv: "1f1ca77b45fb0aff7f2f9b869aae68ca"
server_key_version: "encryptionkey@0"
I20260808 22:31:53.991628 5802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/ts-1/wal/instance:
uuid: "e54274ae7ce2448fb343e0868f28fb06"
format_stamp: "Formatted at 2026-08-08 22:31:53 on dist-test-slave-266d"
server_key: "1d69033fcc2fac9bf93856495b01d4cb"
server_key_iv: "1f1ca77b45fb0aff7f2f9b869aae68ca"
server_key_version: "encryptionkey@0"
I20260808 22:31:53.995167 5802 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 22:31:53.997385 5826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:53.998490 5802 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 22:31:53.998595 5802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/ts-1/wal
uuid: "e54274ae7ce2448fb343e0868f28fb06"
format_stamp: "Formatted at 2026-08-08 22:31:53 on dist-test-slave-266d"
server_key: "1d69033fcc2fac9bf93856495b01d4cb"
server_key_iv: "1f1ca77b45fb0aff7f2f9b869aae68ca"
server_key_version: "encryptionkey@0"
I20260808 22:31:53.998693 5802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:54.011426 5802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:54.014135 5802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:54.014297 5802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:54.014858 5802 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:31:54.015797 5802 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:31:54.015854 5802 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:54.015903 5802 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:31:54.015925 5802 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:54.024151 5802 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.130:44739
I20260808 22:31:54.024171 6006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.130:44739 every 8 connection(s)
I20260808 22:31:54.024964 5802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
I20260808 22:31:54.031544 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 5802
I20260808 22:31:54.031662 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/ts-1/wal/instance
I20260808 22:31:54.031953 32094 external_mini_cluster.cc:1428] Setting key 37432915e60586b1d3127c63712bfee1
Aug 08 22:31:54 dist-test-slave-266d krb5kdc[5399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228313, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.130@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:31:54.038705 5539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:54.026704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:51515 (local address 127.31.87.190:38219)
0808 22:31:54.026842 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:54.026847 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:54.027649 (+ 802us) server_negotiation.cc:407] Connection header received
0808 22:31:54.028926 (+ 1277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:54.028929 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:54.028976 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:54.029047 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:54.030098 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:54.030621 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:54.031859 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:54.032169 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:54.032279 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:54.034900 (+ 2621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:54.034904 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:54.034907 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:54.034935 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:54.036375 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:54.036971 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:54.036974 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:54.036975 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:54.037043 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:54.037445 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:54.037448 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:54.037449 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:54.037625 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:54.037701 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:54.038432 (+ 731us) server_negotiation.cc:299] Negotiation successful
0808 22:31:54.038561 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 22:31:54.039781 6013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:54.027057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:38219 (local address 127.31.87.130:51515)
0808 22:31:54.027496 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:54.027535 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:54.028737 (+ 1202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:54.029212 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:54.029219 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:54.029550 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:54.029986 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:54.029999 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:54.030774 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:54.030779 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:54.031725 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:54.031740 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:54.032318 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:54.032323 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:54.032523 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:54.033138 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:31:54.033161 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:54.034727 (+ 1566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:54.036497 (+ 1770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:54.036504 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:54.036519 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:54.036878 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:54.037161 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:54.037164 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:54.037166 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:54.037355 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:54.037729 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:54.037736 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:54.038072 (+ 336us) client_negotiation.cc:769] Sending connection context
0808 22:31:54.038334 (+ 262us) client_negotiation.cc:240] Negotiation successful
0808 22:31:54.038711 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":118,"threads_started":1}
I20260808 22:31:54.040992 6007 heartbeater.cc:347] Connected to a master server at 127.31.87.190:38219
I20260808 22:31:54.041260 6007 heartbeater.cc:464] Registering TS with master...
I20260808 22:31:54.041739 6007 heartbeater.cc:511] Master 127.31.87.190:38219 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.31.87.131@KRBTEST.COM; defaulting to no policy
I20260808 22:31:54.042817 5474 ts_manager.cc:194] Registered new tserver with Master: e54274ae7ce2448fb343e0868f28fb06 (127.31.87.130:44739)
I20260808 22:31:54.043462 5474 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.130@KRBTEST.COM'} at 127.31.87.130:51515
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab.
Entry for principal kudu/127.31.87.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab.
I20260808 22:31:54.058175 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.131:0
--local_ip_for_outbound_sockets=127.31.87.131
--webserver_interface=127.31.87.131
--webserver_port=0
--tserver_master_addrs=127.31.87.190:38219
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab
--principal=kudu/127.31.87.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:42297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:31:54.164369 6018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:54.164598 6018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:54.164638 6018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:54.168560 6018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:54.168612 6018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:54.168677 6018 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.131
I20260808 22:31:54.173710 6018 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:42297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab
--principal=kudu/127.31.87.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.87.131
--webserver_port=0
--tserver_master_addrs=127.31.87.190:38219
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6018
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.131
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:54.174610 6018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:54.175531 6018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:54.184597 6026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:54.184654 6030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:54.184708 6018 server_base.cc:1034] running on GCE node
W20260808 22:31:54.184798 6025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:54.185242 6018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:54.185815 6018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:54.186975 6018 hybrid_clock.cc:648] HybridClock initialized: now 1786228314186959 us; error 26 us; skew 500 ppm
Aug 08 22:31:54 dist-test-slave-266d krb5kdc[5399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228314, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:31:54.189457 6018 init.cc:377] Logged in from keytab as kudu/127.31.87.131@KRBTEST.COM (short username kudu)
I20260808 22:31:54.190249 6018 webserver.cc:460] Webserver started at http://127.31.87.131:39633/ using document root <none> and password file <none>
I20260808 22:31:54.190738 6018 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:54.190802 6018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:54.190994 6018 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:54.192875 6018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/ts-2/data/instance:
uuid: "1de932166e1a494284bf61795b68cadb"
format_stamp: "Formatted at 2026-08-08 22:31:54 on dist-test-slave-266d"
server_key: "f8e7dbc2bc5ad2c27ed30f9e12b85797"
server_key_iv: "593efe4406d08d402a8fa2e7955bbbe1"
server_key_version: "encryptionkey@0"
I20260808 22:31:54.193450 6018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/ts-2/wal/instance:
uuid: "1de932166e1a494284bf61795b68cadb"
format_stamp: "Formatted at 2026-08-08 22:31:54 on dist-test-slave-266d"
server_key: "f8e7dbc2bc5ad2c27ed30f9e12b85797"
server_key_iv: "593efe4406d08d402a8fa2e7955bbbe1"
server_key_version: "encryptionkey@0"
I20260808 22:31:54.196834 6018 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 22:31:54.199043 6042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:54.200114 6018 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:31:54.200218 6018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/ts-2/wal
uuid: "1de932166e1a494284bf61795b68cadb"
format_stamp: "Formatted at 2026-08-08 22:31:54 on dist-test-slave-266d"
server_key: "f8e7dbc2bc5ad2c27ed30f9e12b85797"
server_key_iv: "593efe4406d08d402a8fa2e7955bbbe1"
server_key_version: "encryptionkey@0"
I20260808 22:31:54.200304 6018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:54.216683 6018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:54.219063 6018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:54.219231 6018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:54.219740 6018 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:31:54.220484 6018 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:31:54.220532 6018 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:54.220582 6018 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:31:54.220613 6018 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:54.229002 6018 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.131:36471
I20260808 22:31:54.229036 6228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.131:36471 every 8 connection(s)
I20260808 22:31:54.229830 6018 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
I20260808 22:31:54.233542 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 6018
I20260808 22:31:54.233670 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786228291013016-32094-0/minicluster-data/ts-2/wal/instance
I20260808 22:31:54.233989 32094 external_mini_cluster.cc:1428] Setting key d2cdf1e89670f8e854f925b438927dbd
Aug 08 22:31:54 dist-test-slave-266d krb5kdc[5399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228314, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.131@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:31:54.242388 5539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:54.231558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:44733 (local address 127.31.87.190:38219)
0808 22:31:54.231724 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:54.231728 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:54.232465 (+ 737us) server_negotiation.cc:407] Connection header received
0808 22:31:54.233818 (+ 1353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:54.233822 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:54.233883 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:54.233977 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:54.235215 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:54.235798 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:54.236937 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:54.237220 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:54.237299 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:54.239522 (+ 2223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:54.239525 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:54.239527 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:54.239551 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:54.240649 (+ 1098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:54.241081 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:54.241084 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:54.241085 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:54.241186 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:54.241450 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:54.241452 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:54.241453 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:54.241612 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:54.241683 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:54.242164 (+ 481us) server_negotiation.cc:299] Negotiation successful
0808 22:31:54.242258 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260808 22:31:54.243141 6233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:54.231875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:38219 (local address 127.31.87.131:44733)
0808 22:31:54.232320 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:54.232360 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:54.233585 (+ 1225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:54.234120 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:54.234127 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:54.234551 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:54.235071 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:54.235087 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:54.235918 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:54.235922 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:54.236829 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:54.236840 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:54.237341 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:54.237344 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:54.237480 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:54.238025 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:31:54.238041 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:54.239363 (+ 1322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:54.240757 (+ 1394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:54.240762 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:54.240773 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:54.241008 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:54.241257 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:54.241259 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:54.241261 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:54.241383 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:54.241729 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:54.241734 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:54.241963 (+ 229us) client_negotiation.cc:769] Sending connection context
0808 22:31:54.242139 (+ 176us) client_negotiation.cc:240] Negotiation successful
0808 22:31:54.242379 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":136,"threads_started":1}
I20260808 22:31:54.244211 6230 heartbeater.cc:347] Connected to a master server at 127.31.87.190:38219
I20260808 22:31:54.244416 6230 heartbeater.cc:464] Registering TS with master...
I20260808 22:31:54.244864 6230 heartbeater.cc:511] Master 127.31.87.190:38219 requested a full tablet report, sending...
I20260808 22:31:54.245934 5474 ts_manager.cc:194] Registered new tserver with Master: 1de932166e1a494284bf61795b68cadb (127.31.87.131:36471)
I20260808 22:31:54.246555 5474 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.131@KRBTEST.COM'} at 127.31.87.131:44733
I20260808 22:31:54.247767 32094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 22:31:54 dist-test-slave-266d krb5kdc[5399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228314, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 22:31:54 dist-test-slave-266d krb5kdc[5399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228314, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.87.129@KRBTEST.COM
I20260808 22:31:54.273046 6239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:54.264491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48488 (local address 127.31.87.129:40393)
0808 22:31:54.264845 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:54.264855 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:54.264879 (+ 24us) server_negotiation.cc:407] Connection header received
0808 22:31:54.264928 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:54.264933 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:54.265124 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:54.265234 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:54.265840 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:54.266593 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:54.267339 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:54.267681 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:54.267755 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:54.269925 (+ 2170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:54.269935 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:54.269949 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:54.269973 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:54.271494 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:54.271895 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:54.271899 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:54.271901 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:54.271973 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:54.272166 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:54.272168 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:54.272169 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:54.272388 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:54.272546 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:54.272674 (+ 128us) server_negotiation.cc:299] Negotiation successful
0808 22:31:54.272834 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":144,"threads_started":1}
Aug 08 22:31:54 dist-test-slave-266d krb5kdc[5399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228314, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:31:54.282927 5539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:54.275865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40258 (local address 127.31.87.190:38219)
0808 22:31:54.275983 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:54.275988 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:54.276029 (+ 41us) server_negotiation.cc:407] Connection header received
0808 22:31:54.276129 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:54.276132 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:54.276170 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:54.276234 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:54.276812 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:54.277416 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:54.278189 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:54.278404 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:54.278477 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:54.280478 (+ 2001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:54.280481 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:54.280483 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:54.280507 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:54.281737 (+ 1230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:54.282061 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:54.282063 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:54.282064 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:54.282128 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:54.282302 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:54.282303 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:54.282304 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:54.282491 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:54.282587 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:54.282719 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 22:31:54.282813 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260808 22:31:54.283308 5474 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40258
I20260808 22:31:54.283833 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 5577
I20260808 22:31:54.302702 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 5802
I20260808 22:31:54.312937 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 6018
I20260808 22:31:54.318853 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 5419
2026-08-08T22:31:54Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2241 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:31:54 dist-test-slave-266d krb5kdc[6247](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:31:54 dist-test-slave-266d krb5kdc[6247](info): set up 2 sockets
Aug 08 22:31:54 dist-test-slave-266d krb5kdc[6247](info): commencing operation
krb5kdc: starting...
W20260808 22:31:55.447196 32094 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Aug 08 22:31:55 dist-test-slave-266d krb5kdc[6247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228315, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:31:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:31:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.87.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
I20260808 22:31:55.557973 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:43539
--webserver_interface=127.31.87.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:33201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:43539
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:31:55.667446 6278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:55.667711 6278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:55.667776 6278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:55.671717 6278 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:31:55.671777 6278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:55.671808 6278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:55.671835 6278 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:31:55.671868 6278 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:31:55.676911 6278 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:33201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:43539
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:43539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.87.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6278
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:55.677902 6278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:55.678838 6278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:55.686637 6290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:55.686671 6283 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:55.686769 6278 server_base.cc:1034] running on GCE node
W20260808 22:31:55.686688 6284 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:55.687373 6278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:55.688359 6278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:55.689517 6278 hybrid_clock.cc:648] HybridClock initialized: now 1786228315689504 us; error 26 us; skew 500 ppm
Aug 08 22:31:55 dist-test-slave-266d krb5kdc[6247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228315, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:31:55.695109 6278 init.cc:377] Logged in from keytab as kudu/127.31.87.190@KRBTEST.COM (short username kudu)
I20260808 22:31:55.695763 6278 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:55.695823 6278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:55.695986 6278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:55.697906 6278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/master-0/data/instance:
uuid: "2bbaaa9437884fd8b251a7beb444d3f2"
format_stamp: "Formatted at 2026-08-08 22:31:55 on dist-test-slave-266d"
server_key: "d2a60db8fea48ce54c415865ff8612b7"
server_key_iv: "5ced897f1e78259bc04f0f6d0f561b95"
server_key_version: "encryptionkey@0"
I20260808 22:31:55.698408 6278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/master-0/wal/instance:
uuid: "2bbaaa9437884fd8b251a7beb444d3f2"
format_stamp: "Formatted at 2026-08-08 22:31:55 on dist-test-slave-266d"
server_key: "d2a60db8fea48ce54c415865ff8612b7"
server_key_iv: "5ced897f1e78259bc04f0f6d0f561b95"
server_key_version: "encryptionkey@0"
I20260808 22:31:55.701727 6278 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:31:55.703944 6303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:55.705032 6278 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:31:55.705183 6278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "2bbaaa9437884fd8b251a7beb444d3f2"
format_stamp: "Formatted at 2026-08-08 22:31:55 on dist-test-slave-266d"
server_key: "d2a60db8fea48ce54c415865ff8612b7"
server_key_iv: "5ced897f1e78259bc04f0f6d0f561b95"
server_key_version: "encryptionkey@0"
I20260808 22:31:55.705271 6278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:55.730809 6278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:55.733805 6278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:55.733950 6278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:55.740278 6278 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.190:43539
I20260808 22:31:55.740512 6384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.190:43539 every 8 connection(s)
I20260808 22:31:55.740876 6278 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:31:55.742913 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 6278
I20260808 22:31:55.743021 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/master-0/wal/instance
I20260808 22:31:55.743279 32094 external_mini_cluster.cc:1428] Setting key f88c2792d48ea6cf666b724fd5ac389d
I20260808 22:31:55.743532 6385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:55.749454 6385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bbaaa9437884fd8b251a7beb444d3f2: Bootstrap starting.
Aug 08 22:31:55 dist-test-slave-266d krb5kdc[6247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228315, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:31:55.751781 6385 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2bbaaa9437884fd8b251a7beb444d3f2: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:55.752501 6385 log.cc:824] T 00000000000000000000000000000000 P 2bbaaa9437884fd8b251a7beb444d3f2: Log is configured to *not* fsync() on all Append() calls
I20260808 22:31:55.754393 6385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bbaaa9437884fd8b251a7beb444d3f2: No bootstrap required, opened a new log
I20260808 22:31:55.756245 6388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:55.744555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46882 (local address 127.31.87.190:43539)
0808 22:31:55.744974 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:55.744989 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:55.745023 (+ 34us) server_negotiation.cc:407] Connection header received
0808 22:31:55.745733 (+ 710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:55.745750 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:55.746087 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:55.746386 (+ 299us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:55.747176 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:55.748140 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:55.748958 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:55.749456 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:55.749609 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:55.752089 (+ 2480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:55.752095 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:55.752110 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:55.752136 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:55.753724 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:55.754117 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:55.754122 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:55.754126 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:55.754233 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:55.754492 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:55.754495 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:55.754497 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:55.754865 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:55.755028 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:55.755327 (+ 299us) server_negotiation.cc:299] Negotiation successful
0808 22:31:55.755587 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":153,"threads_started":1}
I20260808 22:31:55.758033 6385 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2bbaaa9437884fd8b251a7beb444d3f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bbaaa9437884fd8b251a7beb444d3f2" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 43539 } }
I20260808 22:31:55.758283 6385 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2bbaaa9437884fd8b251a7beb444d3f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:55.758356 6385 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2bbaaa9437884fd8b251a7beb444d3f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bbaaa9437884fd8b251a7beb444d3f2, State: Initialized, Role: FOLLOWER
I20260808 22:31:55.758720 6385 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2bbaaa9437884fd8b251a7beb444d3f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bbaaa9437884fd8b251a7beb444d3f2" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 43539 } }
I20260808 22:31:55.758833 6385 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2bbaaa9437884fd8b251a7beb444d3f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:31:55.758893 6385 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2bbaaa9437884fd8b251a7beb444d3f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:31:55.758980 6385 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2bbaaa9437884fd8b251a7beb444d3f2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:55.759739 6385 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2bbaaa9437884fd8b251a7beb444d3f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bbaaa9437884fd8b251a7beb444d3f2" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 43539 } }
I20260808 22:31:55.760032 6385 leader_election.cc:302] T 00000000000000000000000000000000 P 2bbaaa9437884fd8b251a7beb444d3f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2bbaaa9437884fd8b251a7beb444d3f2; no voters:
I20260808 22:31:55.760293 6385 leader_election.cc:288] T 00000000000000000000000000000000 P 2bbaaa9437884fd8b251a7beb444d3f2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:31:55.760368 6392 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2bbaaa9437884fd8b251a7beb444d3f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:31:55.760558 6392 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2bbaaa9437884fd8b251a7beb444d3f2 [term 1 LEADER]: Becoming Leader. State: Replica: 2bbaaa9437884fd8b251a7beb444d3f2, State: Running, Role: LEADER
I20260808 22:31:55.760916 6392 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2bbaaa9437884fd8b251a7beb444d3f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bbaaa9437884fd8b251a7beb444d3f2" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 43539 } }
I20260808 22:31:55.761457 6385 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2bbaaa9437884fd8b251a7beb444d3f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:31:55.762580 6393 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bbaaa9437884fd8b251a7beb444d3f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2bbaaa9437884fd8b251a7beb444d3f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bbaaa9437884fd8b251a7beb444d3f2" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 43539 } } }
I20260808 22:31:55.762567 6396 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bbaaa9437884fd8b251a7beb444d3f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2bbaaa9437884fd8b251a7beb444d3f2. Latest consensus state: current_term: 1 leader_uuid: "2bbaaa9437884fd8b251a7beb444d3f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bbaaa9437884fd8b251a7beb444d3f2" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 43539 } } }
I20260808 22:31:55.762712 6393 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bbaaa9437884fd8b251a7beb444d3f2 [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:55.762712 6396 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bbaaa9437884fd8b251a7beb444d3f2 [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:55.763091 6411 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:31:55.766045 6411 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:31:55.770958 6411 catalog_manager.cc:1347] Generated new cluster ID: afb888c7ffaf43a9902f4a0dc26f5a33
I20260808 22:31:55.771008 6411 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:31:55.784212 6411 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:31:55.785012 6411 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:31:55.795464 6411 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2bbaaa9437884fd8b251a7beb444d3f2: Generated new TSK 0
I20260808 22:31:55.796325 6411 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.87.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab.
Entry for principal kudu/127.31.87.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab.
I20260808 22:31:55.827873 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.129:0
--local_ip_for_outbound_sockets=127.31.87.129
--webserver_interface=127.31.87.129
--webserver_port=0
--tserver_master_addrs=127.31.87.190:43539
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab
--principal=kudu/127.31.87.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:33201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:31:55.938588 6429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:55.938817 6429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:55.938866 6429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:55.942807 6429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:55.942854 6429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:55.942937 6429 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.129
I20260808 22:31:55.947908 6429 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:33201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab
--principal=kudu/127.31.87.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.87.129
--webserver_port=0
--tserver_master_addrs=127.31.87.190:43539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6429
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.129
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:55.948837 6429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:55.949770 6429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:55.958467 6435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:55.958467 6436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:55.958647 6429 server_base.cc:1034] running on GCE node
W20260808 22:31:55.958550 6438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:55.959103 6429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:55.959667 6429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:55.960826 6429 hybrid_clock.cc:648] HybridClock initialized: now 1786228315960807 us; error 33 us; skew 500 ppm
Aug 08 22:31:55 dist-test-slave-266d krb5kdc[6247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228315, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:31:55.963625 6429 init.cc:377] Logged in from keytab as kudu/127.31.87.129@KRBTEST.COM (short username kudu)
I20260808 22:31:55.964272 6429 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:55.964339 6429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:55.964519 6429 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:55.966420 6429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/ts-0/data/instance:
uuid: "e878464cd83f4c8d85cb791ea35548a9"
format_stamp: "Formatted at 2026-08-08 22:31:55 on dist-test-slave-266d"
server_key: "63a3cbd25942b5365c6f5637bb906f34"
server_key_iv: "990a689929042c61419a50be0e2e94ad"
server_key_version: "encryptionkey@0"
I20260808 22:31:55.966933 6429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/ts-0/wal/instance:
uuid: "e878464cd83f4c8d85cb791ea35548a9"
format_stamp: "Formatted at 2026-08-08 22:31:55 on dist-test-slave-266d"
server_key: "63a3cbd25942b5365c6f5637bb906f34"
server_key_iv: "990a689929042c61419a50be0e2e94ad"
server_key_version: "encryptionkey@0"
I20260808 22:31:55.970299 6429 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:31:55.972416 6447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:55.973551 6429 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:31:55.973650 6429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/ts-0/wal
uuid: "e878464cd83f4c8d85cb791ea35548a9"
format_stamp: "Formatted at 2026-08-08 22:31:55 on dist-test-slave-266d"
server_key: "63a3cbd25942b5365c6f5637bb906f34"
server_key_iv: "990a689929042c61419a50be0e2e94ad"
server_key_version: "encryptionkey@0"
I20260808 22:31:55.973733 6429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:55.997256 6429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:56.000164 6429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:56.000331 6429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:56.000850 6429 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:31:56.001646 6429 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:31:56.001693 6429 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:56.001742 6429 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:31:56.001770 6429 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:56.009883 6429 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.129:44801
I20260808 22:31:56.009919 6635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.129:44801 every 8 connection(s)
I20260808 22:31:56.010444 6429 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
I20260808 22:31:56.013729 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 6429
I20260808 22:31:56.013852 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/ts-0/wal/instance
I20260808 22:31:56.014163 32094 external_mini_cluster.cc:1428] Setting key 4989e1f873689f1c76457c1d91ba451e
Aug 08 22:31:56 dist-test-slave-266d krb5kdc[6247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228315, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.129@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:31:56.024346 6388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:56.012007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:36921 (local address 127.31.87.190:43539)
0808 22:31:56.012128 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:56.012132 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:56.012934 (+ 802us) server_negotiation.cc:407] Connection header received
0808 22:31:56.014505 (+ 1571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:56.014508 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:56.014573 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:56.014675 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:56.015992 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:56.016549 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:56.017993 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:56.018338 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:56.018439 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:56.020766 (+ 2327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:56.020771 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:56.020773 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:56.020803 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:56.022375 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:56.022842 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:56.022845 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:56.022847 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:56.022945 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:56.023229 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:56.023233 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:56.023234 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:56.023470 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:56.023576 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:56.024015 (+ 439us) server_negotiation.cc:299] Negotiation successful
0808 22:31:56.024157 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
WARNING: no policy specified for kudu/127.31.87.130@KRBTEST.COM; defaulting to no policy
I20260808 22:31:56.024950 6638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:56.012333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:43539 (local address 127.31.87.129:36921)
0808 22:31:56.012785 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:56.012827 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:56.014280 (+ 1453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:56.014838 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:56.014846 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:56.015211 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:56.015838 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:56.015859 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:56.016680 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:56.016684 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:56.017876 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:56.017886 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:56.018488 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:56.018491 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:56.018647 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:56.019170 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:31:56.019186 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:56.020591 (+ 1405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:56.022483 (+ 1892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:56.022487 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:56.022499 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:56.022748 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:56.023025 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:56.023027 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:56.023028 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:56.023142 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:56.023558 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:56.023562 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:56.023788 (+ 226us) client_negotiation.cc:769] Sending connection context
0808 22:31:56.023933 (+ 145us) client_negotiation.cc:240] Negotiation successful
0808 22:31:56.024157 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":120,"threads_started":1}
I20260808 22:31:56.026160 6636 heartbeater.cc:347] Connected to a master server at 127.31.87.190:43539
I20260808 22:31:56.026374 6636 heartbeater.cc:464] Registering TS with master...
I20260808 22:31:56.026811 6636 heartbeater.cc:511] Master 127.31.87.190:43539 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.130@KRBTEST.COM" created.
I20260808 22:31:56.028223 6330 ts_manager.cc:194] Registered new tserver with Master: e878464cd83f4c8d85cb791ea35548a9 (127.31.87.129:44801)
I20260808 22:31:56.029383 6330 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.129@KRBTEST.COM'} at 127.31.87.129:36921
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab.
Entry for principal kudu/127.31.87.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab.
I20260808 22:31:56.041477 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.130:0
--local_ip_for_outbound_sockets=127.31.87.130
--webserver_interface=127.31.87.130
--webserver_port=0
--tserver_master_addrs=127.31.87.190:43539
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab
--principal=kudu/127.31.87.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:33201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:31:56.150187 6641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:56.150415 6641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:56.150460 6641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:56.154413 6641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:56.154466 6641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:56.154522 6641 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.130
I20260808 22:31:56.159587 6641 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:33201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab
--principal=kudu/127.31.87.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.87.130
--webserver_port=0
--tserver_master_addrs=127.31.87.190:43539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6641
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.130
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:56.160533 6641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:56.161526 6641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:56.170660 6646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:56.170686 6649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:56.170660 6651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:56.170794 6641 server_base.cc:1034] running on GCE node
I20260808 22:31:56.171378 6641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:56.171974 6641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:56.173147 6641 hybrid_clock.cc:648] HybridClock initialized: now 1786228316173129 us; error 29 us; skew 500 ppm
Aug 08 22:31:56 dist-test-slave-266d krb5kdc[6247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228316, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:31:56.175669 6641 init.cc:377] Logged in from keytab as kudu/127.31.87.130@KRBTEST.COM (short username kudu)
I20260808 22:31:56.176396 6641 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:56.176466 6641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:56.176680 6641 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:56.178678 6641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/ts-1/data/instance:
uuid: "d7982a66fe1d4faa9bfbca00bb7ee925"
format_stamp: "Formatted at 2026-08-08 22:31:56 on dist-test-slave-266d"
server_key: "d7b678417547be5f45348fa9c4868413"
server_key_iv: "18428f06640fcaa4d64f80cb382aa1de"
server_key_version: "encryptionkey@0"
I20260808 22:31:56.179196 6641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/ts-1/wal/instance:
uuid: "d7982a66fe1d4faa9bfbca00bb7ee925"
format_stamp: "Formatted at 2026-08-08 22:31:56 on dist-test-slave-266d"
server_key: "d7b678417547be5f45348fa9c4868413"
server_key_iv: "18428f06640fcaa4d64f80cb382aa1de"
server_key_version: "encryptionkey@0"
I20260808 22:31:56.182574 6641 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20260808 22:31:56.184755 6661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:56.185880 6641 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 22:31:56.185992 6641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/ts-1/wal
uuid: "d7982a66fe1d4faa9bfbca00bb7ee925"
format_stamp: "Formatted at 2026-08-08 22:31:56 on dist-test-slave-266d"
server_key: "d7b678417547be5f45348fa9c4868413"
server_key_iv: "18428f06640fcaa4d64f80cb382aa1de"
server_key_version: "encryptionkey@0"
I20260808 22:31:56.186079 6641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:56.202937 6641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:56.205910 6641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:56.206072 6641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:56.206573 6641 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:31:56.207365 6641 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:31:56.207410 6641 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:56.207453 6641 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:31:56.207480 6641 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:56.215461 6641 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.130:40369
I20260808 22:31:56.215510 6840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.130:40369 every 8 connection(s)
I20260808 22:31:56.215974 6641 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
I20260808 22:31:56.217340 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 6641
I20260808 22:31:56.217478 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/ts-1/wal/instance
I20260808 22:31:56.217795 32094 external_mini_cluster.cc:1428] Setting key fd9c526b5f6d94756f1ea583eeacae39
Aug 08 22:31:56 dist-test-slave-266d krb5kdc[6247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228316, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.130@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.87.131@KRBTEST.COM; defaulting to no policy
I20260808 22:31:56.228945 6388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:56.219010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:45731 (local address 127.31.87.190:43539)
0808 22:31:56.219122 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:56.219126 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:56.219138 (+ 12us) server_negotiation.cc:407] Connection header received
0808 22:31:56.220083 (+ 945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:56.220087 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:56.220140 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:56.220211 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:56.221427 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:56.221959 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:56.223181 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:56.223406 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:56.223494 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:56.225756 (+ 2262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:56.225761 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:56.225764 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:56.225792 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:56.227205 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:56.227671 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:56.227675 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:56.227677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:56.227770 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:56.228045 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:56.228047 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:56.228048 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:56.228222 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:56.228295 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:56.228721 (+ 426us) server_negotiation.cc:299] Negotiation successful
0808 22:31:56.228825 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 22:31:56.229773 6844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:56.218104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:43539 (local address 127.31.87.130:45731)
0808 22:31:56.218572 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:56.218626 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:56.219886 (+ 1260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:56.220386 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:56.220393 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:56.220714 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:56.221274 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:56.221294 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:56.222080 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:56.222084 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:56.223062 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:56.223073 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:56.223542 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:56.223547 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:56.223720 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:56.224224 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:31:56.224240 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:56.225593 (+ 1353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:56.227316 (+ 1723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:56.227320 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:56.227331 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:56.227589 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:56.227857 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:56.227859 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:56.227860 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:56.227978 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:56.228294 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:56.228298 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:56.228530 (+ 232us) client_negotiation.cc:769] Sending connection context
0808 22:31:56.228675 (+ 145us) client_negotiation.cc:240] Negotiation successful
0808 22:31:56.228920 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":107,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.131@KRBTEST.COM" created.
I20260808 22:31:56.230883 6842 heartbeater.cc:347] Connected to a master server at 127.31.87.190:43539
I20260808 22:31:56.231106 6842 heartbeater.cc:464] Registering TS with master...
I20260808 22:31:56.231503 6842 heartbeater.cc:511] Master 127.31.87.190:43539 requested a full tablet report, sending...
I20260808 22:31:56.232484 6330 ts_manager.cc:194] Registered new tserver with Master: d7982a66fe1d4faa9bfbca00bb7ee925 (127.31.87.130:40369)
I20260808 22:31:56.233031 6330 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.130@KRBTEST.COM'} at 127.31.87.130:45731
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab.
Entry for principal kudu/127.31.87.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab.
I20260808 22:31:56.244344 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.131:0
--local_ip_for_outbound_sockets=127.31.87.131
--webserver_interface=127.31.87.131
--webserver_port=0
--tserver_master_addrs=127.31.87.190:43539
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab
--principal=kudu/127.31.87.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:33201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:31:56.356874 6850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:56.357165 6850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:56.357214 6850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:56.361186 6850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:56.361240 6850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:56.361297 6850 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.131
I20260808 22:31:56.366510 6850 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:33201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab
--principal=kudu/127.31.87.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.87.131
--webserver_port=0
--tserver_master_addrs=127.31.87.190:43539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6850
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.131
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:56.367650 6850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:56.368585 6850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:56.377609 6858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:56.377566 6862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:56.377749 6860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:56.377838 6850 server_base.cc:1034] running on GCE node
I20260808 22:31:56.378253 6850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:56.378856 6850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:56.380009 6850 hybrid_clock.cc:648] HybridClock initialized: now 1786228316379992 us; error 27 us; skew 500 ppm
Aug 08 22:31:56 dist-test-slave-266d krb5kdc[6247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228316, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:31:56.382597 6850 init.cc:377] Logged in from keytab as kudu/127.31.87.131@KRBTEST.COM (short username kudu)
I20260808 22:31:56.383319 6850 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:56.383387 6850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:56.383577 6850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:56.385540 6850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/ts-2/data/instance:
uuid: "159a2f11168c45dfa1ed4860897b3ffb"
format_stamp: "Formatted at 2026-08-08 22:31:56 on dist-test-slave-266d"
server_key: "ea46cf123813f0d367ee4001c09c0758"
server_key_iv: "72c0d005ab5a0efc852526f8492f2ae1"
server_key_version: "encryptionkey@0"
I20260808 22:31:56.386065 6850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/ts-2/wal/instance:
uuid: "159a2f11168c45dfa1ed4860897b3ffb"
format_stamp: "Formatted at 2026-08-08 22:31:56 on dist-test-slave-266d"
server_key: "ea46cf123813f0d367ee4001c09c0758"
server_key_iv: "72c0d005ab5a0efc852526f8492f2ae1"
server_key_version: "encryptionkey@0"
I20260808 22:31:56.389562 6850 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:31:56.391857 6877 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:56.392946 6850 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:31:56.393047 6850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/ts-2/wal
uuid: "159a2f11168c45dfa1ed4860897b3ffb"
format_stamp: "Formatted at 2026-08-08 22:31:56 on dist-test-slave-266d"
server_key: "ea46cf123813f0d367ee4001c09c0758"
server_key_iv: "72c0d005ab5a0efc852526f8492f2ae1"
server_key_version: "encryptionkey@0"
I20260808 22:31:56.393164 6850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:56.411168 6850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:56.413682 6850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:56.413834 6850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:56.414328 6850 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:31:56.415071 6850 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:31:56.415115 6850 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:56.415165 6850 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:31:56.415195 6850 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:56.423326 6850 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.131:45559
I20260808 22:31:56.423348 7052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.131:45559 every 8 connection(s)
I20260808 22:31:56.423828 6850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
I20260808 22:31:56.430620 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 6850
I20260808 22:31:56.430728 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/ts-2/wal/instance
I20260808 22:31:56.430975 32094 external_mini_cluster.cc:1428] Setting key c06ce5381239daf94dc46a2beab62d72
Aug 08 22:31:56 dist-test-slave-266d krb5kdc[6247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228316, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.131@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:31:56.435837 6388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:56.425334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:54137 (local address 127.31.87.190:43539)
0808 22:31:56.425469 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:56.425473 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:56.426208 (+ 735us) server_negotiation.cc:407] Connection header received
0808 22:31:56.427471 (+ 1263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:56.427475 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:56.427532 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:56.427601 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:56.428663 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:56.429225 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:56.430295 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:56.430531 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:56.430620 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:56.433033 (+ 2413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:56.433036 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:56.433039 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:56.433063 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:56.434297 (+ 1234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:56.434732 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:56.434734 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:56.434735 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:56.434802 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:56.435041 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:56.435043 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:56.435044 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:56.435194 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:56.435260 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:56.435627 (+ 367us) server_negotiation.cc:299] Negotiation successful
0808 22:31:56.435719 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 22:31:56.436545 7057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:56.425655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:43539 (local address 127.31.87.131:54137)
0808 22:31:56.426079 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:56.426116 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:56.427275 (+ 1159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:56.427775 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:56.427782 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:56.428102 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:56.428526 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:56.428539 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:56.429343 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:56.429346 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:56.430169 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:56.430179 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:56.430665 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:56.430669 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:56.430854 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:56.431327 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:31:56.431342 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:56.432875 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:56.434392 (+ 1517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:56.434397 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:56.434407 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:56.434651 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:56.434861 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:56.434863 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:56.434865 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:56.434979 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:56.435259 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:56.435263 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:56.435459 (+ 196us) client_negotiation.cc:769] Sending connection context
0808 22:31:56.435597 (+ 138us) client_negotiation.cc:240] Negotiation successful
0808 22:31:56.435816 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":114,"threads_started":1}
I20260808 22:31:56.437582 7055 heartbeater.cc:347] Connected to a master server at 127.31.87.190:43539
I20260808 22:31:56.437781 7055 heartbeater.cc:464] Registering TS with master...
I20260808 22:31:56.438156 7055 heartbeater.cc:511] Master 127.31.87.190:43539 requested a full tablet report, sending...
I20260808 22:31:56.439036 6330 ts_manager.cc:194] Registered new tserver with Master: 159a2f11168c45dfa1ed4860897b3ffb (127.31.87.131:45559)
I20260808 22:31:56.439543 6330 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.131@KRBTEST.COM'} at 127.31.87.131:54137
I20260808 22:31:56.444278 32094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:31:56.451958 6388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:56.445993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46888 (local address 127.31.87.190:43539)
0808 22:31:56.446131 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:56.446135 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:56.446284 (+ 149us) server_negotiation.cc:407] Connection header received
0808 22:31:56.446374 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:56.446376 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:56.446413 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:56.446472 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:56.447027 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:56.447494 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:56.448208 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:56.448405 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:56.448470 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:56.449631 (+ 1161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:56.449634 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:56.449636 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:56.449657 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:56.450747 (+ 1090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:56.451080 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:56.451082 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:56.451083 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:56.451144 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:56.451315 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:56.451317 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:56.451318 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:56.451498 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:56.451596 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:56.451732 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 22:31:56.451816 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 22:31:56.454432 6330 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46888:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:31:56.457329 6330 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:31:56.470605 7070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:56.465043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:44801 (local address 127.0.0.1:38052)
0808 22:31:56.465368 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:56.465410 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:56.465526 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:56.466071 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:56.466077 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:56.466102 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:31:56.466228 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:56.466234 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:56.467746 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:56.467750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:56.468916 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:56.468926 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:56.470080 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:56.470083 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:56.470192 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:56.470230 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 22:31:56.470352 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 22:31:56.470466 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":85,"threads_started":1}
I20260808 22:31:56.470633 7073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:56.465228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38052 (local address 127.31.87.129:44801)
0808 22:31:56.465568 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:56.465580 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:56.465604 (+ 24us) server_negotiation.cc:407] Connection header received
0808 22:31:56.465686 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:56.465691 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:56.465905 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:56.466070 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:31:56.466370 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:56.467605 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:56.469162 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:56.469961 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:56.470072 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:56.470219 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:56.470390 (+ 171us) server_negotiation.cc:299] Negotiation successful
0808 22:31:56.470482 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":140,"threads_started":1}
I20260808 22:31:56.471161 7076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:56.465757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.131:45559 (local address 127.0.0.1:44864)
0808 22:31:56.466781 (+ 1024us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:56.466794 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:56.466880 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:56.467350 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:56.467353 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:56.467371 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:31:56.467453 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:56.467458 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:56.468419 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:56.468422 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:56.469655 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:56.469666 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:56.470792 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:56.470795 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:56.470876 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:56.470898 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 22:31:56.470971 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 22:31:56.471037 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":967,"mutex_wait_us":42,"thread_start_us":81,"threads_started":1}
I20260808 22:31:56.471271 7080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:56.466001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44864 (local address 127.31.87.131:45559)
0808 22:31:56.466649 (+ 648us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:56.466660 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:56.466964 (+ 304us) server_negotiation.cc:407] Connection header received
0808 22:31:56.467035 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:56.467040 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:56.467209 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:56.467355 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:31:56.467552 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:56.468301 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:56.469804 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:56.470500 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:56.470590 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:56.470716 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:56.471020 (+ 304us) server_negotiation.cc:299] Negotiation successful
0808 22:31:56.471109 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":118,"threads_started":1}
I20260808 22:31:56.471462 7081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:56.466114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35666 (local address 127.31.87.130:40369)
0808 22:31:56.466411 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:56.466423 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:56.466710 (+ 287us) server_negotiation.cc:407] Connection header received
0808 22:31:56.466795 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:56.466800 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:56.466992 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:56.467129 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:31:56.467382 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:56.468461 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:56.469831 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:56.470591 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:56.470709 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:56.470843 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:56.471228 (+ 385us) server_negotiation.cc:299] Negotiation successful
0808 22:31:56.471322 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":123,"threads_started":1}
I20260808 22:31:56.471706 7074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:56.465699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.130:40369 (local address 127.0.0.1:35666)
0808 22:31:56.466498 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:56.466510 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:56.466601 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:56.467122 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:56.467125 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:56.467142 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:31:56.467258 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:56.467265 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:56.468585 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:56.468588 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:56.469733 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:56.469740 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:56.470903 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:56.470906 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:56.470996 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:56.471020 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 22:31:56.471089 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 22:31:56.471152 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":739,"mutex_wait_us":2,"thread_start_us":102,"threads_started":1}
I20260808 22:31:56.473928 6529 tablet_service.cc:1467] Processing CreateTablet for tablet 6df49d67be2d48f98bfa5c5db10ae2d3 (DEFAULT_TABLE table=test-table [id=193a8524495e41c7b6a7bf34ceccbb0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:31:56.474357 6746 tablet_service.cc:1467] Processing CreateTablet for tablet 6df49d67be2d48f98bfa5c5db10ae2d3 (DEFAULT_TABLE table=test-table [id=193a8524495e41c7b6a7bf34ceccbb0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:31:56.474615 6949 tablet_service.cc:1467] Processing CreateTablet for tablet 6df49d67be2d48f98bfa5c5db10ae2d3 (DEFAULT_TABLE table=test-table [id=193a8524495e41c7b6a7bf34ceccbb0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:31:56.475063 6529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6df49d67be2d48f98bfa5c5db10ae2d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:56.475317 6746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6df49d67be2d48f98bfa5c5db10ae2d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:56.475508 6949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6df49d67be2d48f98bfa5c5db10ae2d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:56.481647 7082 tablet_bootstrap.cc:492] T 6df49d67be2d48f98bfa5c5db10ae2d3 P 159a2f11168c45dfa1ed4860897b3ffb: Bootstrap starting.
I20260808 22:31:56.482817 7084 tablet_bootstrap.cc:492] T 6df49d67be2d48f98bfa5c5db10ae2d3 P d7982a66fe1d4faa9bfbca00bb7ee925: Bootstrap starting.
I20260808 22:31:56.483709 7082 tablet_bootstrap.cc:654] T 6df49d67be2d48f98bfa5c5db10ae2d3 P 159a2f11168c45dfa1ed4860897b3ffb: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:56.484416 7084 tablet_bootstrap.cc:654] T 6df49d67be2d48f98bfa5c5db10ae2d3 P d7982a66fe1d4faa9bfbca00bb7ee925: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:56.484862 7082 log.cc:824] T 6df49d67be2d48f98bfa5c5db10ae2d3 P 159a2f11168c45dfa1ed4860897b3ffb: Log is configured to *not* fsync() on all Append() calls
I20260808 22:31:56.485394 7083 tablet_bootstrap.cc:492] T 6df49d67be2d48f98bfa5c5db10ae2d3 P e878464cd83f4c8d85cb791ea35548a9: Bootstrap starting.
I20260808 22:31:56.485570 7084 log.cc:824] T 6df49d67be2d48f98bfa5c5db10ae2d3 P d7982a66fe1d4faa9bfbca00bb7ee925: Log is configured to *not* fsync() on all Append() calls
I20260808 22:31:56.486726 7082 tablet_bootstrap.cc:492] T 6df49d67be2d48f98bfa5c5db10ae2d3 P 159a2f11168c45dfa1ed4860897b3ffb: No bootstrap required, opened a new log
I20260808 22:31:56.486949 7082 ts_tablet_manager.cc:1397] T 6df49d67be2d48f98bfa5c5db10ae2d3 P 159a2f11168c45dfa1ed4860897b3ffb: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260808 22:31:56.487377 7083 tablet_bootstrap.cc:654] T 6df49d67be2d48f98bfa5c5db10ae2d3 P e878464cd83f4c8d85cb791ea35548a9: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:56.488267 7084 tablet_bootstrap.cc:492] T 6df49d67be2d48f98bfa5c5db10ae2d3 P d7982a66fe1d4faa9bfbca00bb7ee925: No bootstrap required, opened a new log
I20260808 22:31:56.488446 7084 ts_tablet_manager.cc:1397] T 6df49d67be2d48f98bfa5c5db10ae2d3 P d7982a66fe1d4faa9bfbca00bb7ee925: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260808 22:31:56.488596 7083 log.cc:824] T 6df49d67be2d48f98bfa5c5db10ae2d3 P e878464cd83f4c8d85cb791ea35548a9: Log is configured to *not* fsync() on all Append() calls
I20260808 22:31:56.490072 7082 raft_consensus.cc:348] T 6df49d67be2d48f98bfa5c5db10ae2d3 P 159a2f11168c45dfa1ed4860897b3ffb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e878464cd83f4c8d85cb791ea35548a9" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 44801 } } peers { permanent_uuid: "159a2f11168c45dfa1ed4860897b3ffb" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45559 } } peers { permanent_uuid: "d7982a66fe1d4faa9bfbca00bb7ee925" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 40369 } }
I20260808 22:31:56.490320 7082 raft_consensus.cc:374] T 6df49d67be2d48f98bfa5c5db10ae2d3 P 159a2f11168c45dfa1ed4860897b3ffb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:56.490394 7082 raft_consensus.cc:729] T 6df49d67be2d48f98bfa5c5db10ae2d3 P 159a2f11168c45dfa1ed4860897b3ffb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 159a2f11168c45dfa1ed4860897b3ffb, State: Initialized, Role: FOLLOWER
I20260808 22:31:56.490513 7083 tablet_bootstrap.cc:492] T 6df49d67be2d48f98bfa5c5db10ae2d3 P e878464cd83f4c8d85cb791ea35548a9: No bootstrap required, opened a new log
I20260808 22:31:56.490686 7083 ts_tablet_manager.cc:1397] T 6df49d67be2d48f98bfa5c5db10ae2d3 P e878464cd83f4c8d85cb791ea35548a9: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 22:31:56.490867 7082 consensus_queue.cc:260] T 6df49d67be2d48f98bfa5c5db10ae2d3 P 159a2f11168c45dfa1ed4860897b3ffb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e878464cd83f4c8d85cb791ea35548a9" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 44801 } } peers { permanent_uuid: "159a2f11168c45dfa1ed4860897b3ffb" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45559 } } peers { permanent_uuid: "d7982a66fe1d4faa9bfbca00bb7ee925" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 40369 } }
I20260808 22:31:56.491430 7055 heartbeater.cc:503] Master 127.31.87.190:43539 was elected leader, sending a full tablet report...
I20260808 22:31:56.491783 7082 ts_tablet_manager.cc:1428] T 6df49d67be2d48f98bfa5c5db10ae2d3 P 159a2f11168c45dfa1ed4860897b3ffb: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260808 22:31:56.492092 7084 raft_consensus.cc:348] T 6df49d67be2d48f98bfa5c5db10ae2d3 P d7982a66fe1d4faa9bfbca00bb7ee925 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e878464cd83f4c8d85cb791ea35548a9" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 44801 } } peers { permanent_uuid: "159a2f11168c45dfa1ed4860897b3ffb" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45559 } } peers { permanent_uuid: "d7982a66fe1d4faa9bfbca00bb7ee925" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 40369 } }
I20260808 22:31:56.492374 7084 raft_consensus.cc:374] T 6df49d67be2d48f98bfa5c5db10ae2d3 P d7982a66fe1d4faa9bfbca00bb7ee925 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:56.492460 7084 raft_consensus.cc:729] T 6df49d67be2d48f98bfa5c5db10ae2d3 P d7982a66fe1d4faa9bfbca00bb7ee925 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7982a66fe1d4faa9bfbca00bb7ee925, State: Initialized, Role: FOLLOWER
I20260808 22:31:56.492911 7084 consensus_queue.cc:260] T 6df49d67be2d48f98bfa5c5db10ae2d3 P d7982a66fe1d4faa9bfbca00bb7ee925 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e878464cd83f4c8d85cb791ea35548a9" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 44801 } } peers { permanent_uuid: "159a2f11168c45dfa1ed4860897b3ffb" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45559 } } peers { permanent_uuid: "d7982a66fe1d4faa9bfbca00bb7ee925" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 40369 } }
I20260808 22:31:56.493902 7084 ts_tablet_manager.cc:1428] T 6df49d67be2d48f98bfa5c5db10ae2d3 P d7982a66fe1d4faa9bfbca00bb7ee925: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 22:31:56.494210 6842 heartbeater.cc:503] Master 127.31.87.190:43539 was elected leader, sending a full tablet report...
I20260808 22:31:56.494467 7083 raft_consensus.cc:348] T 6df49d67be2d48f98bfa5c5db10ae2d3 P e878464cd83f4c8d85cb791ea35548a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e878464cd83f4c8d85cb791ea35548a9" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 44801 } } peers { permanent_uuid: "159a2f11168c45dfa1ed4860897b3ffb" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45559 } } peers { permanent_uuid: "d7982a66fe1d4faa9bfbca00bb7ee925" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 40369 } }
I20260808 22:31:56.494679 7083 raft_consensus.cc:374] T 6df49d67be2d48f98bfa5c5db10ae2d3 P e878464cd83f4c8d85cb791ea35548a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:56.494724 7083 raft_consensus.cc:729] T 6df49d67be2d48f98bfa5c5db10ae2d3 P e878464cd83f4c8d85cb791ea35548a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e878464cd83f4c8d85cb791ea35548a9, State: Initialized, Role: FOLLOWER
I20260808 22:31:56.495090 7083 consensus_queue.cc:260] T 6df49d67be2d48f98bfa5c5db10ae2d3 P e878464cd83f4c8d85cb791ea35548a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e878464cd83f4c8d85cb791ea35548a9" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 44801 } } peers { permanent_uuid: "159a2f11168c45dfa1ed4860897b3ffb" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45559 } } peers { permanent_uuid: "d7982a66fe1d4faa9bfbca00bb7ee925" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 40369 } }
I20260808 22:31:56.496166 6636 heartbeater.cc:503] Master 127.31.87.190:43539 was elected leader, sending a full tablet report...
I20260808 22:31:56.496449 7083 ts_tablet_manager.cc:1428] T 6df49d67be2d48f98bfa5c5db10ae2d3 P e878464cd83f4c8d85cb791ea35548a9: Time spent starting tablet: real 0.006s user 0.000s sys 0.006s
W20260808 22:31:56.511499 6637 tablet.cc:2369] T 6df49d67be2d48f98bfa5c5db10ae2d3 P e878464cd83f4c8d85cb791ea35548a9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:31:56.574373 7095 raft_consensus.cc:482] T 6df49d67be2d48f98bfa5c5db10ae2d3 P e878464cd83f4c8d85cb791ea35548a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:31:56.574561 7095 raft_consensus.cc:504] T 6df49d67be2d48f98bfa5c5db10ae2d3 P e878464cd83f4c8d85cb791ea35548a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e878464cd83f4c8d85cb791ea35548a9" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 44801 } } peers { permanent_uuid: "159a2f11168c45dfa1ed4860897b3ffb" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45559 } } peers { permanent_uuid: "d7982a66fe1d4faa9bfbca00bb7ee925" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 40369 } }
I20260808 22:31:56.575675 7095 leader_election.cc:288] T 6df49d67be2d48f98bfa5c5db10ae2d3 P e878464cd83f4c8d85cb791ea35548a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 159a2f11168c45dfa1ed4860897b3ffb (127.31.87.131:45559), d7982a66fe1d4faa9bfbca00bb7ee925 (127.31.87.130:40369)
I20260808 22:31:56.579535 7080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:56.576012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:51623 (local address 127.31.87.131:45559)
0808 22:31:56.576149 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:56.576154 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:56.576414 (+ 260us) server_negotiation.cc:407] Connection header received
0808 22:31:56.576542 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:56.576546 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:56.576597 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:56.576670 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:31:56.576917 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:56.577552 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:56.578599 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:56.578984 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:56.579090 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:56.579141 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:56.579351 (+ 210us) server_negotiation.cc:299] Negotiation successful
0808 22:31:56.579416 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 22:31:56.579540 7081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:56.576093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:45369 (local address 127.31.87.130:40369)
0808 22:31:56.576223 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:56.576227 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:56.576316 (+ 89us) server_negotiation.cc:407] Connection header received
0808 22:31:56.576488 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:56.576492 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:56.576543 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:56.576612 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:31:56.576910 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:56.577508 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:56.578580 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:56.578968 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:56.579076 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:56.579133 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:56.579347 (+ 214us) server_negotiation.cc:299] Negotiation successful
0808 22:31:56.579418 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 22:31:56.579558 7099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:56.575876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.130:40369 (local address 127.31.87.129:45369)
0808 22:31:56.576223 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:56.576238 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:56.576358 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:56.576623 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:56.576626 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:56.576643 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:31:56.576799 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:56.576804 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:56.577604 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:56.577629 (+ 25us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:56.578462 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:56.578472 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:56.579085 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:56.579089 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:56.579205 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:56.579256 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 22:31:56.579319 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 22:31:56.579403 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"mutex_wait_us":24,"thread_start_us":97,"threads_started":1}
I20260808 22:31:56.579564 7096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:56.575876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.131:45559 (local address 127.31.87.129:51623)
0808 22:31:56.576325 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:56.576337 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:56.576434 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:56.576691 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:56.576694 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:56.576711 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:31:56.576812 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:56.576817 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:56.577671 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:56.577674 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:56.578478 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:56.578487 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:56.579092 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:56.579096 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:56.579216 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:56.579259 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 22:31:56.579324 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 22:31:56.579418 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"mutex_wait_us":113,"thread_start_us":134,"threads_started":1}
I20260808 22:31:56.580149 6777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6df49d67be2d48f98bfa5c5db10ae2d3" candidate_uuid: "e878464cd83f4c8d85cb791ea35548a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7982a66fe1d4faa9bfbca00bb7ee925" is_pre_election: true
I20260808 22:31:56.580276 6977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6df49d67be2d48f98bfa5c5db10ae2d3" candidate_uuid: "e878464cd83f4c8d85cb791ea35548a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "159a2f11168c45dfa1ed4860897b3ffb" is_pre_election: true
I20260808 22:31:56.580442 6777 raft_consensus.cc:2454] T 6df49d67be2d48f98bfa5c5db10ae2d3 P d7982a66fe1d4faa9bfbca00bb7ee925 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e878464cd83f4c8d85cb791ea35548a9 in term 0.
I20260808 22:31:56.580531 6977 raft_consensus.cc:2454] T 6df49d67be2d48f98bfa5c5db10ae2d3 P 159a2f11168c45dfa1ed4860897b3ffb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e878464cd83f4c8d85cb791ea35548a9 in term 0.
I20260808 22:31:56.580873 6449 leader_election.cc:302] T 6df49d67be2d48f98bfa5c5db10ae2d3 P e878464cd83f4c8d85cb791ea35548a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7982a66fe1d4faa9bfbca00bb7ee925, e878464cd83f4c8d85cb791ea35548a9; no voters:
I20260808 22:31:56.581142 7095 raft_consensus.cc:2781] T 6df49d67be2d48f98bfa5c5db10ae2d3 P e878464cd83f4c8d85cb791ea35548a9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:31:56.581220 7095 raft_consensus.cc:482] T 6df49d67be2d48f98bfa5c5db10ae2d3 P e878464cd83f4c8d85cb791ea35548a9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:31:56.581264 7095 raft_consensus.cc:3037] T 6df49d67be2d48f98bfa5c5db10ae2d3 P e878464cd83f4c8d85cb791ea35548a9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:56.582257 7095 raft_consensus.cc:504] T 6df49d67be2d48f98bfa5c5db10ae2d3 P e878464cd83f4c8d85cb791ea35548a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e878464cd83f4c8d85cb791ea35548a9" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 44801 } } peers { permanent_uuid: "159a2f11168c45dfa1ed4860897b3ffb" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45559 } } peers { permanent_uuid: "d7982a66fe1d4faa9bfbca00bb7ee925" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 40369 } }
I20260808 22:31:56.582580 7095 leader_election.cc:288] T 6df49d67be2d48f98bfa5c5db10ae2d3 P e878464cd83f4c8d85cb791ea35548a9 [CANDIDATE]: Term 1 election: Requested vote from peers 159a2f11168c45dfa1ed4860897b3ffb (127.31.87.131:45559), d7982a66fe1d4faa9bfbca00bb7ee925 (127.31.87.130:40369)
I20260808 22:31:56.582911 6977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6df49d67be2d48f98bfa5c5db10ae2d3" candidate_uuid: "e878464cd83f4c8d85cb791ea35548a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "159a2f11168c45dfa1ed4860897b3ffb"
I20260808 22:31:56.582911 6777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6df49d67be2d48f98bfa5c5db10ae2d3" candidate_uuid: "e878464cd83f4c8d85cb791ea35548a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7982a66fe1d4faa9bfbca00bb7ee925"
I20260808 22:31:56.583029 6977 raft_consensus.cc:3037] T 6df49d67be2d48f98bfa5c5db10ae2d3 P 159a2f11168c45dfa1ed4860897b3ffb [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:56.583029 6777 raft_consensus.cc:3037] T 6df49d67be2d48f98bfa5c5db10ae2d3 P d7982a66fe1d4faa9bfbca00bb7ee925 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:56.584281 6977 raft_consensus.cc:2454] T 6df49d67be2d48f98bfa5c5db10ae2d3 P 159a2f11168c45dfa1ed4860897b3ffb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e878464cd83f4c8d85cb791ea35548a9 in term 1.
I20260808 22:31:56.584281 6777 raft_consensus.cc:2454] T 6df49d67be2d48f98bfa5c5db10ae2d3 P d7982a66fe1d4faa9bfbca00bb7ee925 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e878464cd83f4c8d85cb791ea35548a9 in term 1.
I20260808 22:31:56.584568 6449 leader_election.cc:302] T 6df49d67be2d48f98bfa5c5db10ae2d3 P e878464cd83f4c8d85cb791ea35548a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 159a2f11168c45dfa1ed4860897b3ffb, d7982a66fe1d4faa9bfbca00bb7ee925, e878464cd83f4c8d85cb791ea35548a9; no voters:
I20260808 22:31:56.584741 7095 raft_consensus.cc:2781] T 6df49d67be2d48f98bfa5c5db10ae2d3 P e878464cd83f4c8d85cb791ea35548a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:31:56.584954 7095 raft_consensus.cc:686] T 6df49d67be2d48f98bfa5c5db10ae2d3 P e878464cd83f4c8d85cb791ea35548a9 [term 1 LEADER]: Becoming Leader. State: Replica: e878464cd83f4c8d85cb791ea35548a9, State: Running, Role: LEADER
I20260808 22:31:56.585304 7095 consensus_queue.cc:237] T 6df49d67be2d48f98bfa5c5db10ae2d3 P e878464cd83f4c8d85cb791ea35548a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e878464cd83f4c8d85cb791ea35548a9" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 44801 } } peers { permanent_uuid: "159a2f11168c45dfa1ed4860897b3ffb" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45559 } } peers { permanent_uuid: "d7982a66fe1d4faa9bfbca00bb7ee925" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 40369 } }
I20260808 22:31:56.588133 6330 catalog_manager.cc:5495] T 6df49d67be2d48f98bfa5c5db10ae2d3 P e878464cd83f4c8d85cb791ea35548a9 reported cstate change: term changed from 0 to 1, leader changed from <none> to e878464cd83f4c8d85cb791ea35548a9 (127.31.87.129). New cstate: current_term: 1 leader_uuid: "e878464cd83f4c8d85cb791ea35548a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e878464cd83f4c8d85cb791ea35548a9" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 44801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "159a2f11168c45dfa1ed4860897b3ffb" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7982a66fe1d4faa9bfbca00bb7ee925" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 40369 } health_report { overall_health: UNKNOWN } } }
I20260808 22:31:56.619421 7073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:56.615949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38064 (local address 127.31.87.129:44801)
0808 22:31:56.616083 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:56.616087 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:56.616101 (+ 14us) server_negotiation.cc:407] Connection header received
0808 22:31:56.616253 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:56.616255 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:56.616322 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:56.616413 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:31:56.616673 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:56.617199 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:56.618032 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:56.618252 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:56.618341 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:56.618642 (+ 301us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:31:56.619045 (+ 403us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:31:56.619149 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:56.619231 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 22:31:56.619288 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 22:31:56.625013 6977 raft_consensus.cc:1261] T 6df49d67be2d48f98bfa5c5db10ae2d3 P 159a2f11168c45dfa1ed4860897b3ffb [term 1 FOLLOWER]: Refusing update from remote peer e878464cd83f4c8d85cb791ea35548a9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:31:56.625013 6777 raft_consensus.cc:1261] T 6df49d67be2d48f98bfa5c5db10ae2d3 P d7982a66fe1d4faa9bfbca00bb7ee925 [term 1 FOLLOWER]: Refusing update from remote peer e878464cd83f4c8d85cb791ea35548a9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:31:56.625754 7095 consensus_queue.cc:1035] T 6df49d67be2d48f98bfa5c5db10ae2d3 P e878464cd83f4c8d85cb791ea35548a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "159a2f11168c45dfa1ed4860897b3ffb" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:31:56.625945 7100 consensus_queue.cc:1035] T 6df49d67be2d48f98bfa5c5db10ae2d3 P e878464cd83f4c8d85cb791ea35548a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7982a66fe1d4faa9bfbca00bb7ee925" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 40369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:31:56.633307 7111 mvcc.cc:205] Tried to move back new op lower bound from 7316391184889061376 to 7316391184735817728. Current Snapshot: MvccSnapshot[applied={T|T < 7316391184889061376}]
I20260808 22:31:56.635387 7112 mvcc.cc:205] Tried to move back new op lower bound from 7316391184889061376 to 7316391184735817728. Current Snapshot: MvccSnapshot[applied={T|T < 7316391184889061376 or (T in {7316391184889061376})}]
I20260808 22:31:56.640441 6330 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46888:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:31:56.640604 6330 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46888:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:31:56.642959 6330 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2bbaaa9437884fd8b251a7beb444d3f2: Sending DeleteTablet for 3 replicas of tablet 6df49d67be2d48f98bfa5c5db10ae2d3
I20260808 22:31:56.643733 6746 tablet_service.cc:1514] Processing DeleteTablet for tablet 6df49d67be2d48f98bfa5c5db10ae2d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:31:56 UTC) from {username='kudu', principal='kudu/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:35666
I20260808 22:31:56.643733 6529 tablet_service.cc:1514] Processing DeleteTablet for tablet 6df49d67be2d48f98bfa5c5db10ae2d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:31:56 UTC) from {username='kudu', principal='kudu/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:38052
I20260808 22:31:56.644107 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 6429
I20260808 22:31:56.644196 6949 tablet_service.cc:1514] Processing DeleteTablet for tablet 6df49d67be2d48f98bfa5c5db10ae2d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:31:56 UTC) from {username='kudu', principal='kudu/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:44864
I20260808 22:31:56.649791 7124 tablet_replica.cc:330] T 6df49d67be2d48f98bfa5c5db10ae2d3 P 159a2f11168c45dfa1ed4860897b3ffb: stopping tablet replica
I20260808 22:31:56.649931 7123 tablet_replica.cc:330] T 6df49d67be2d48f98bfa5c5db10ae2d3 P d7982a66fe1d4faa9bfbca00bb7ee925: stopping tablet replica
I20260808 22:31:56.650190 7123 raft_consensus.cc:2229] T 6df49d67be2d48f98bfa5c5db10ae2d3 P d7982a66fe1d4faa9bfbca00bb7ee925 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:31:56.650190 7124 raft_consensus.cc:2229] T 6df49d67be2d48f98bfa5c5db10ae2d3 P 159a2f11168c45dfa1ed4860897b3ffb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:31:56.650499 7123 raft_consensus.cc:2258] T 6df49d67be2d48f98bfa5c5db10ae2d3 P d7982a66fe1d4faa9bfbca00bb7ee925 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:31:56.650498 7124 raft_consensus.cc:2258] T 6df49d67be2d48f98bfa5c5db10ae2d3 P 159a2f11168c45dfa1ed4860897b3ffb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:31:56.654059 7124 ts_tablet_manager.cc:1905] T 6df49d67be2d48f98bfa5c5db10ae2d3 P 159a2f11168c45dfa1ed4860897b3ffb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:31:56.654059 7123 ts_tablet_manager.cc:1905] T 6df49d67be2d48f98bfa5c5db10ae2d3 P d7982a66fe1d4faa9bfbca00bb7ee925: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:31:56.656805 7124 ts_tablet_manager.cc:1918] T 6df49d67be2d48f98bfa5c5db10ae2d3 P 159a2f11168c45dfa1ed4860897b3ffb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:31:56.656838 7123 ts_tablet_manager.cc:1918] T 6df49d67be2d48f98bfa5c5db10ae2d3 P d7982a66fe1d4faa9bfbca00bb7ee925: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:31:56.656912 7123 log.cc:1196] T 6df49d67be2d48f98bfa5c5db10ae2d3 P d7982a66fe1d4faa9bfbca00bb7ee925: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/ts-1/wal/wals/6df49d67be2d48f98bfa5c5db10ae2d3
I20260808 22:31:56.656912 7124 log.cc:1196] T 6df49d67be2d48f98bfa5c5db10ae2d3 P 159a2f11168c45dfa1ed4860897b3ffb: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786228291013016-32094-0/minicluster-data/ts-2/wal/wals/6df49d67be2d48f98bfa5c5db10ae2d3
I20260808 22:31:56.657274 7124 ts_tablet_manager.cc:1939] T 6df49d67be2d48f98bfa5c5db10ae2d3 P 159a2f11168c45dfa1ed4860897b3ffb: Deleting consensus metadata
I20260808 22:31:56.657301 7123 ts_tablet_manager.cc:1939] T 6df49d67be2d48f98bfa5c5db10ae2d3 P d7982a66fe1d4faa9bfbca00bb7ee925: Deleting consensus metadata
I20260808 22:31:56.658509 6308 catalog_manager.cc:4841] TS 159a2f11168c45dfa1ed4860897b3ffb (127.31.87.131:45559): tablet 6df49d67be2d48f98bfa5c5db10ae2d3 (table test-table [id=193a8524495e41c7b6a7bf34ceccbb0b]) successfully deleted
I20260808 22:31:56.658600 6304 catalog_manager.cc:4841] TS d7982a66fe1d4faa9bfbca00bb7ee925 (127.31.87.130:40369): tablet 6df49d67be2d48f98bfa5c5db10ae2d3 (table test-table [id=193a8524495e41c7b6a7bf34ceccbb0b]) successfully deleted
W20260808 22:31:56.658805 6308 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.87.129:44801 (error 108)
I20260808 22:31:56.658917 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 6641
I20260808 22:31:56.669569 7070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:56.659655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:44801 (local address 127.0.0.1:38072)
0808 22:31:56.669377 (+ 9722us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:56.669471 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.129:44801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":9614}
W20260808 22:31:56.670605 6308 catalog_manager.cc:4584] TS e878464cd83f4c8d85cb791ea35548a9 (127.31.87.129:44801): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6df49d67be2d48f98bfa5c5db10ae2d3: Network error: Client connection negotiation failed: client connection to 127.31.87.129:44801: connect: Connection refused (error 111)
I20260808 22:31:56.675455 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 6850
I20260808 22:31:56.682677 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 6278
2026-08-08T22:31:56Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2363 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-08T22:31:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:31:56Z Disabled control of system clock
I20260808 22:31:56.716269 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:37163
--webserver_interface=127.31.87.190
--webserver_port=0
--builtin_ntp_servers=127.31.87.148:45541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:37163
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 22:31:56.822698 7134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:56.822934 7134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:56.822973 7134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:56.826869 7134 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:31:56.826920 7134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:56.826934 7134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:56.826964 7134 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:31:56.826992 7134 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:31:56.831576 7134 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:45541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:37163
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:37163
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.87.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:56.832548 7134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:56.833877 7134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:56.841962 7141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:56.842061 7134 server_base.cc:1034] running on GCE node
W20260808 22:31:56.841986 7144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:56.841962 7140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:56.842651 7134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:56.843639 7134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:56.844790 7134 hybrid_clock.cc:648] HybridClock initialized: now 1786228316844777 us; error 24 us; skew 500 ppm
I20260808 22:31:56.846526 7134 webserver.cc:460] Webserver started at http://127.31.87.190:41625/ using document root <none> and password file <none>
I20260808 22:31:56.847048 7134 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:56.847107 7134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:56.847299 7134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:56.849267 7134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/master-0/data/instance:
uuid: "28bbee1727ee4adda5a3cffccc322c73"
format_stamp: "Formatted at 2026-08-08 22:31:56 on dist-test-slave-266d"
server_key: "cbf874c6882bb290bddc8e141357b348"
server_key_iv: "1cbf307071f5ac9a3036eba1a2d37233"
server_key_version: "encryptionkey@0"
I20260808 22:31:56.849792 7134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/master-0/wal/instance:
uuid: "28bbee1727ee4adda5a3cffccc322c73"
format_stamp: "Formatted at 2026-08-08 22:31:56 on dist-test-slave-266d"
server_key: "cbf874c6882bb290bddc8e141357b348"
server_key_iv: "1cbf307071f5ac9a3036eba1a2d37233"
server_key_version: "encryptionkey@0"
I20260808 22:31:56.853354 7134 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:31:56.855523 7150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:56.856674 7134 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:31:56.856789 7134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "28bbee1727ee4adda5a3cffccc322c73"
format_stamp: "Formatted at 2026-08-08 22:31:56 on dist-test-slave-266d"
server_key: "cbf874c6882bb290bddc8e141357b348"
server_key_iv: "1cbf307071f5ac9a3036eba1a2d37233"
server_key_version: "encryptionkey@0"
I20260808 22:31:56.856879 7134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:56.861172 7134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:56.861717 7134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:56.861877 7134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:56.868148 7134 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.190:37163
I20260808 22:31:56.868153 7229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.190:37163 every 8 connection(s)
I20260808 22:31:56.868983 7134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:31:56.870869 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 7134
I20260808 22:31:56.870981 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/master-0/wal/instance
I20260808 22:31:56.871291 32094 external_mini_cluster.cc:1428] Setting key e1d25eeca20198ba97f6a43e397d9962
I20260808 22:31:56.872483 7230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:56.877079 7235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:56.872568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35182 (local address 127.31.87.190:37163)
0808 22:31:56.873183 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:56.873198 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:56.873237 (+ 39us) server_negotiation.cc:407] Connection header received
0808 22:31:56.873852 (+ 615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:56.873866 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:56.874183 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:56.874445 (+ 262us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:56.875362 (+ 917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:56.875368 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:56.875384 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:31:56.875411 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:56.875474 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:31:56.875495 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:56.875628 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:56.875966 (+ 338us) server_negotiation.cc:299] Negotiation successful
0808 22:31:56.876167 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":107,"threads_started":1}
I20260808 22:31:56.878818 7230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28bbee1727ee4adda5a3cffccc322c73: Bootstrap starting.
I20260808 22:31:56.880932 7230 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 28bbee1727ee4adda5a3cffccc322c73: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:56.881735 7230 log.cc:824] T 00000000000000000000000000000000 P 28bbee1727ee4adda5a3cffccc322c73: Log is configured to *not* fsync() on all Append() calls
I20260808 22:31:56.883452 7230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28bbee1727ee4adda5a3cffccc322c73: No bootstrap required, opened a new log
I20260808 22:31:56.886711 7230 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28bbee1727ee4adda5a3cffccc322c73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28bbee1727ee4adda5a3cffccc322c73" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 37163 } }
I20260808 22:31:56.886914 7230 raft_consensus.cc:374] T 00000000000000000000000000000000 P 28bbee1727ee4adda5a3cffccc322c73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:56.886970 7230 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28bbee1727ee4adda5a3cffccc322c73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bbee1727ee4adda5a3cffccc322c73, State: Initialized, Role: FOLLOWER
I20260808 22:31:56.887355 7230 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28bbee1727ee4adda5a3cffccc322c73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28bbee1727ee4adda5a3cffccc322c73" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 37163 } }
I20260808 22:31:56.887486 7230 raft_consensus.cc:388] T 00000000000000000000000000000000 P 28bbee1727ee4adda5a3cffccc322c73 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:31:56.887549 7230 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28bbee1727ee4adda5a3cffccc322c73 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:31:56.887645 7230 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28bbee1727ee4adda5a3cffccc322c73 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:56.888427 7230 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28bbee1727ee4adda5a3cffccc322c73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28bbee1727ee4adda5a3cffccc322c73" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 37163 } }
I20260808 22:31:56.888763 7230 leader_election.cc:302] T 00000000000000000000000000000000 P 28bbee1727ee4adda5a3cffccc322c73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 28bbee1727ee4adda5a3cffccc322c73; no voters:
I20260808 22:31:56.889041 7230 leader_election.cc:288] T 00000000000000000000000000000000 P 28bbee1727ee4adda5a3cffccc322c73 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:31:56.889139 7237 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28bbee1727ee4adda5a3cffccc322c73 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:31:56.889323 7237 raft_consensus.cc:686] T 00000000000000000000000000000000 P 28bbee1727ee4adda5a3cffccc322c73 [term 1 LEADER]: Becoming Leader. State: Replica: 28bbee1727ee4adda5a3cffccc322c73, State: Running, Role: LEADER
I20260808 22:31:56.889597 7237 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28bbee1727ee4adda5a3cffccc322c73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28bbee1727ee4adda5a3cffccc322c73" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 37163 } }
I20260808 22:31:56.890259 7230 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28bbee1727ee4adda5a3cffccc322c73 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:31:56.891283 7241 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28bbee1727ee4adda5a3cffccc322c73 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28bbee1727ee4adda5a3cffccc322c73. Latest consensus state: current_term: 1 leader_uuid: "28bbee1727ee4adda5a3cffccc322c73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28bbee1727ee4adda5a3cffccc322c73" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 37163 } } }
I20260808 22:31:56.891307 7238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28bbee1727ee4adda5a3cffccc322c73 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "28bbee1727ee4adda5a3cffccc322c73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28bbee1727ee4adda5a3cffccc322c73" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 37163 } } }
I20260808 22:31:56.891498 7238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28bbee1727ee4adda5a3cffccc322c73 [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:56.891793 7241 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28bbee1727ee4adda5a3cffccc322c73 [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:56.891911 7248 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:31:56.894927 7248 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:31:56.895231 7260 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 22:31:56.900192 7248 catalog_manager.cc:1347] Generated new cluster ID: a17a3dc92a6c4d39a106f68e8e479f37
I20260808 22:31:56.900254 7248 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:31:56.908387 7248 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:31:56.908648 7248 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:31:56.912267 7248 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 28bbee1727ee4adda5a3cffccc322c73: Generated new TSK 0
I20260808 22:31:56.912935 7248 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:31:56.921319 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.129:0
--local_ip_for_outbound_sockets=127.31.87.129
--webserver_interface=127.31.87.129
--webserver_port=0
--tserver_master_addrs=127.31.87.190:37163
--builtin_ntp_servers=127.31.87.148:45541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 22:31:57.028703 7262 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:57.028935 7262 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:57.028977 7262 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:57.032866 7262 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:57.032909 7262 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:57.032975 7262 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.129
I20260808 22:31:57.037530 7262 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:45541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.87.129
--webserver_port=0
--tserver_master_addrs=127.31.87.190:37163
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7262
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.129
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:57.038434 7262 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:57.039672 7262 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:57.048570 7275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:57.048575 7268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:57.048646 7262 server_base.cc:1034] running on GCE node
W20260808 22:31:57.048575 7270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:57.049213 7262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:57.049784 7262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:57.050936 7262 hybrid_clock.cc:648] HybridClock initialized: now 1786228317050919 us; error 28 us; skew 500 ppm
I20260808 22:31:57.052574 7262 webserver.cc:460] Webserver started at http://127.31.87.129:46691/ using document root <none> and password file <none>
I20260808 22:31:57.053125 7262 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:57.053193 7262 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:57.053395 7262 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:57.055246 7262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/ts-0/data/instance:
uuid: "9269f9cb31504d418d330020b13c816a"
format_stamp: "Formatted at 2026-08-08 22:31:57 on dist-test-slave-266d"
server_key: "cb5b8c8bc9a2c12ce1f12d1df0d9aa24"
server_key_iv: "0ba108ad73d35b3c5f9c562a76cd104a"
server_key_version: "encryptionkey@0"
I20260808 22:31:57.055750 7262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/ts-0/wal/instance:
uuid: "9269f9cb31504d418d330020b13c816a"
format_stamp: "Formatted at 2026-08-08 22:31:57 on dist-test-slave-266d"
server_key: "cb5b8c8bc9a2c12ce1f12d1df0d9aa24"
server_key_iv: "0ba108ad73d35b3c5f9c562a76cd104a"
server_key_version: "encryptionkey@0"
I20260808 22:31:57.059092 7262 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 22:31:57.061446 7284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:57.062539 7262 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:31:57.062640 7262 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/ts-0/wal
uuid: "9269f9cb31504d418d330020b13c816a"
format_stamp: "Formatted at 2026-08-08 22:31:57 on dist-test-slave-266d"
server_key: "cb5b8c8bc9a2c12ce1f12d1df0d9aa24"
server_key_iv: "0ba108ad73d35b3c5f9c562a76cd104a"
server_key_version: "encryptionkey@0"
I20260808 22:31:57.062728 7262 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:57.066866 7262 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:57.067386 7262 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:57.067530 7262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:57.068005 7262 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:31:57.068817 7262 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:31:57.068872 7262 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:57.068920 7262 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:31:57.068944 7262 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:57.076591 7262 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.129:37307
I20260808 22:31:57.076642 7454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.129:37307 every 8 connection(s)
I20260808 22:31:57.077397 7262 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
I20260808 22:31:57.082677 7235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:57.078920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:34655 (local address 127.31.87.190:37163)
0808 22:31:57.079078 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:57.079083 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:57.080020 (+ 937us) server_negotiation.cc:407] Connection header received
0808 22:31:57.081336 (+ 1316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:57.081339 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:57.081378 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:57.081447 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:57.082347 (+ 900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:57.082350 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:57.082352 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:31:57.082363 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:57.082382 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:31:57.082389 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:57.082457 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:57.082538 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 22:31:57.082573 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 22:31:57.083424 7457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:57.079287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37163 (local address 127.31.87.129:34655)
0808 22:31:57.079884 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:57.079920 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:57.081161 (+ 1241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:57.081581 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:57.081587 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:57.081679 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:57.082123 (+ 444us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:31:57.082125 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:57.082161 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:31:57.082224 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:57.082448 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:57.082452 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:57.082456 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:31:57.082568 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 22:31:57.082751 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":103,"threads_started":1}
I20260808 22:31:57.084456 7455 heartbeater.cc:347] Connected to a master server at 127.31.87.190:37163
I20260808 22:31:57.084638 7455 heartbeater.cc:464] Registering TS with master...
I20260808 22:31:57.084961 7455 heartbeater.cc:511] Master 127.31.87.190:37163 requested a full tablet report, sending...
I20260808 22:31:57.086418 7171 ts_manager.cc:194] Registered new tserver with Master: 9269f9cb31504d418d330020b13c816a (127.31.87.129:37307)
I20260808 22:31:57.087450 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 7262
I20260808 22:31:57.087569 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/ts-0/wal/instance
I20260808 22:31:57.087815 32094 external_mini_cluster.cc:1428] Setting key e171a6a1e388eb06cbdb0737daf3800e
I20260808 22:31:57.089553 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.130:0
--local_ip_for_outbound_sockets=127.31.87.130
--webserver_interface=127.31.87.130
--webserver_port=0
--tserver_master_addrs=127.31.87.190:37163
--builtin_ntp_servers=127.31.87.148:45541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 22:31:57.196918 7458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:57.197172 7458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:57.197212 7458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:57.201051 7458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:57.201120 7458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:57.201197 7458 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.130
I20260808 22:31:57.205715 7458 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:45541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.87.130
--webserver_port=0
--tserver_master_addrs=127.31.87.190:37163
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7458
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.130
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:57.206612 7458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:57.207881 7458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 22:31:57.217000 7458 server_base.cc:1034] running on GCE node
W20260808 22:31:57.216948 7467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:57.217022 7468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:57.216962 7471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:57.217626 7458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:57.218225 7458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:57.219389 7458 hybrid_clock.cc:648] HybridClock initialized: now 1786228317219362 us; error 37 us; skew 500 ppm
I20260808 22:31:57.221108 7458 webserver.cc:460] Webserver started at http://127.31.87.130:39925/ using document root <none> and password file <none>
I20260808 22:31:57.221663 7458 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:57.221735 7458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:57.221946 7458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:57.223878 7458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/ts-1/data/instance:
uuid: "2cfbe7cfb2e54457b49d48363aa69c91"
format_stamp: "Formatted at 2026-08-08 22:31:57 on dist-test-slave-266d"
server_key: "eb2e164a7efe2dc30fc7f5edaa2bdaa0"
server_key_iv: "24f3a56b0dcef6c6a43da6f22754620e"
server_key_version: "encryptionkey@0"
I20260808 22:31:57.224421 7458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/ts-1/wal/instance:
uuid: "2cfbe7cfb2e54457b49d48363aa69c91"
format_stamp: "Formatted at 2026-08-08 22:31:57 on dist-test-slave-266d"
server_key: "eb2e164a7efe2dc30fc7f5edaa2bdaa0"
server_key_iv: "24f3a56b0dcef6c6a43da6f22754620e"
server_key_version: "encryptionkey@0"
I20260808 22:31:57.228047 7458 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.001s
I20260808 22:31:57.230530 7480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:57.231678 7458 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:31:57.231801 7458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/ts-1/wal
uuid: "2cfbe7cfb2e54457b49d48363aa69c91"
format_stamp: "Formatted at 2026-08-08 22:31:57 on dist-test-slave-266d"
server_key: "eb2e164a7efe2dc30fc7f5edaa2bdaa0"
server_key_iv: "24f3a56b0dcef6c6a43da6f22754620e"
server_key_version: "encryptionkey@0"
I20260808 22:31:57.231904 7458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:57.236480 7458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:57.237128 7458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:57.237308 7458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:57.237957 7458 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:31:57.238893 7458 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:31:57.238945 7458 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:57.238992 7458 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:31:57.239018 7458 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:57.247215 7458 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.130:33965
I20260808 22:31:57.247268 7647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.130:33965 every 8 connection(s)
I20260808 22:31:57.248039 7458 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
I20260808 22:31:57.253569 7235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:57.249733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:59029 (local address 127.31.87.190:37163)
0808 22:31:57.249870 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:57.249873 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:57.250864 (+ 991us) server_negotiation.cc:407] Connection header received
0808 22:31:57.252089 (+ 1225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:57.252092 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:57.252132 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:57.252211 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:57.253212 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:57.253215 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:57.253217 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:31:57.253229 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:57.253253 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:31:57.253260 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:57.253333 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:57.253428 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 22:31:57.253463 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 22:31:57.254348 7650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:57.250108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37163 (local address 127.31.87.130:59029)
0808 22:31:57.250729 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:57.250766 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:57.251880 (+ 1114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:57.252344 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:57.252350 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:57.252434 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:57.252917 (+ 483us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:31:57.252919 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:57.252956 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:31:57.253022 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:57.253327 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:57.253332 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:57.253336 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:31:57.253458 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 22:31:57.253641 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":108,"threads_started":1}
I20260808 22:31:57.255308 7648 heartbeater.cc:347] Connected to a master server at 127.31.87.190:37163
I20260808 22:31:57.255481 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 7458
I20260808 22:31:57.255533 7648 heartbeater.cc:464] Registering TS with master...
I20260808 22:31:57.255633 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/ts-1/wal/instance
I20260808 22:31:57.255950 32094 external_mini_cluster.cc:1428] Setting key c1043c6054d407e925eddfc78001f08a
I20260808 22:31:57.255963 7648 heartbeater.cc:511] Master 127.31.87.190:37163 requested a full tablet report, sending...
I20260808 22:31:57.256940 7170 ts_manager.cc:194] Registered new tserver with Master: 2cfbe7cfb2e54457b49d48363aa69c91 (127.31.87.130:33965)
I20260808 22:31:57.257835 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.131:0
--local_ip_for_outbound_sockets=127.31.87.131
--webserver_interface=127.31.87.131
--webserver_port=0
--tserver_master_addrs=127.31.87.190:37163
--builtin_ntp_servers=127.31.87.148:45541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 22:31:57.365497 7651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:57.365758 7651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:57.365798 7651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:57.369735 7651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:57.369784 7651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:57.369840 7651 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.131
I20260808 22:31:57.374440 7651 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:45541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.87.131
--webserver_port=0
--tserver_master_addrs=127.31.87.190:37163
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7651
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.131
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:57.375356 7651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:57.376757 7651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:57.386034 7661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:57.386036 7666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:57.386018 7662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:57.386287 7651 server_base.cc:1034] running on GCE node
I20260808 22:31:57.386708 7651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:57.387347 7651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:57.388504 7651 hybrid_clock.cc:648] HybridClock initialized: now 1786228317388472 us; error 47 us; skew 500 ppm
I20260808 22:31:57.390338 7651 webserver.cc:460] Webserver started at http://127.31.87.131:45435/ using document root <none> and password file <none>
I20260808 22:31:57.390980 7651 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:57.391039 7651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:57.391245 7651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:57.393424 7651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/ts-2/data/instance:
uuid: "a8ae993042fe4a16ae5f6d958b9f8eab"
format_stamp: "Formatted at 2026-08-08 22:31:57 on dist-test-slave-266d"
server_key: "1fc3b464784392619c745a22188f0f10"
server_key_iv: "27e3b9666688170ba8557171af6641c0"
server_key_version: "encryptionkey@0"
I20260808 22:31:57.393953 7651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/ts-2/wal/instance:
uuid: "a8ae993042fe4a16ae5f6d958b9f8eab"
format_stamp: "Formatted at 2026-08-08 22:31:57 on dist-test-slave-266d"
server_key: "1fc3b464784392619c745a22188f0f10"
server_key_iv: "27e3b9666688170ba8557171af6641c0"
server_key_version: "encryptionkey@0"
I20260808 22:31:57.397552 7651 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:31:57.399895 7675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:57.401059 7651 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:31:57.401194 7651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/ts-2/wal
uuid: "a8ae993042fe4a16ae5f6d958b9f8eab"
format_stamp: "Formatted at 2026-08-08 22:31:57 on dist-test-slave-266d"
server_key: "1fc3b464784392619c745a22188f0f10"
server_key_iv: "27e3b9666688170ba8557171af6641c0"
server_key_version: "encryptionkey@0"
I20260808 22:31:57.401288 7651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:57.405522 7651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:57.406165 7651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:57.406340 7651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:57.406860 7651 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:31:57.407747 7651 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:31:57.407804 7651 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:57.407855 7651 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:31:57.407884 7651 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:57.416278 7651 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.131:43887
I20260808 22:31:57.416319 7838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.131:43887 every 8 connection(s)
I20260808 22:31:57.417070 7651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
I20260808 22:31:57.422240 7235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:57.418606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:50913 (local address 127.31.87.190:37163)
0808 22:31:57.418756 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:57.418760 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:57.419703 (+ 943us) server_negotiation.cc:407] Connection header received
0808 22:31:57.420867 (+ 1164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:57.420870 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:57.420904 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:57.420971 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:57.421919 (+ 948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:57.421922 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:57.421923 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:31:57.421935 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:57.421954 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:31:57.421961 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:57.422027 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:57.422105 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 22:31:57.422138 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 22:31:57.422999 7843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:57.418978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37163 (local address 127.31.87.131:50913)
0808 22:31:57.419564 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:57.419610 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:57.420707 (+ 1097us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:57.421135 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:57.421142 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:57.421226 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:57.421683 (+ 457us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:31:57.421685 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:57.421721 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:31:57.421784 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:57.422015 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:57.422019 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:57.422022 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 22:31:57.422136 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 22:31:57.422312 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":101,"threads_started":1}
I20260808 22:31:57.424014 7839 heartbeater.cc:347] Connected to a master server at 127.31.87.190:37163
I20260808 22:31:57.424209 7839 heartbeater.cc:464] Registering TS with master...
I20260808 22:31:57.424232 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 7651
I20260808 22:31:57.424338 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/ts-2/wal/instance
I20260808 22:31:57.424579 32094 external_mini_cluster.cc:1428] Setting key 35e99e4e5269b84bb65e700832a5253a
I20260808 22:31:57.424582 7839 heartbeater.cc:511] Master 127.31.87.190:37163 requested a full tablet report, sending...
I20260808 22:31:57.425534 7170 ts_manager.cc:194] Registered new tserver with Master: a8ae993042fe4a16ae5f6d958b9f8eab (127.31.87.131:43887)
I20260808 22:31:57.426465 32094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:31:57.429617 7235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:57.428339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35194 (local address 127.31.87.190:37163)
0808 22:31:57.428435 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:57.428438 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:57.428645 (+ 207us) server_negotiation.cc:407] Connection header received
0808 22:31:57.428739 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:57.428741 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:57.428768 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:57.428829 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:57.429366 (+ 537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:57.429369 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:57.429371 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:31:57.429381 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:57.429396 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:31:57.429402 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:57.429463 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:57.429502 (+ 39us) server_negotiation.cc:299] Negotiation successful
0808 22:31:57.429531 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 22:31:57.431975 7171 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35194:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:31:57.434823 7171 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:31:57.445611 7863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:57.442820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.131:43887 (local address 127.0.0.1:38792)
0808 22:31:57.443477 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:57.443489 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:57.443550 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:57.444142 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:57.444147 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:57.444219 (+ 72us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:57.444824 (+ 605us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:31:57.444826 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:57.444837 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:31:57.444871 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:57.445413 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:57.445416 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:57.445419 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 22:31:57.445481 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 22:31:57.445507 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"thread_start_us":74,"threads_started":1}
I20260808 22:31:57.445616 7865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:57.443169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50590 (local address 127.31.87.130:33965)
0808 22:31:57.443752 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:57.443764 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:57.443786 (+ 22us) server_negotiation.cc:407] Connection header received
0808 22:31:57.443848 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:57.443854 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:57.444152 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:57.444480 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:57.445029 (+ 549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:57.445033 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:57.445042 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:31:57.445049 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:57.445081 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:31:57.445141 (+ 60us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:57.445243 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:57.445378 (+ 135us) server_negotiation.cc:299] Negotiation successful
0808 22:31:57.445451 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":144,"threads_started":1}
I20260808 22:31:57.445732 7862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:57.442537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41420 (local address 127.31.87.129:37307)
0808 22:31:57.442965 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:57.442978 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:57.443250 (+ 272us) server_negotiation.cc:407] Connection header received
0808 22:31:57.443459 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:57.443467 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:57.443666 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:57.443789 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:57.445176 (+ 1387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:57.445182 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:57.445193 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:31:57.445205 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:57.445249 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:31:57.445283 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:57.445374 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:57.445535 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 22:31:57.445595 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":116,"threads_started":1}
I20260808 22:31:57.446256 7864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:57.443143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38792 (local address 127.31.87.131:43887)
0808 22:31:57.443754 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:57.443764 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:57.443785 (+ 21us) server_negotiation.cc:407] Connection header received
0808 22:31:57.443828 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:57.443833 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:57.443991 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:57.444115 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:57.445188 (+ 1073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:57.445193 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:57.445204 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:31:57.445214 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:57.445251 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:31:57.445287 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:57.445374 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:57.446070 (+ 696us) server_negotiation.cc:299] Negotiation successful
0808 22:31:57.446125 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":519,"thread_start_us":91,"threads_started":1}
I20260808 22:31:57.445739 7859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:57.442561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.130:33965 (local address 127.0.0.1:50590)
0808 22:31:57.443113 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:57.443160 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:57.443251 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:57.444294 (+ 1043us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:57.444296 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:57.444309 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:57.444782 (+ 473us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:31:57.444784 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:57.444804 (+ 20us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:31:57.444869 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:57.445223 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:57.445232 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:57.445237 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 22:31:57.445354 (+ 117us) client_negotiation.cc:240] Negotiation successful
0808 22:31:57.445419 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":82,"threads_started":1}
I20260808 22:31:57.446105 7858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:57.442401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:37307 (local address 127.0.0.1:41420)
0808 22:31:57.443111 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:57.443155 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:57.443328 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:57.444457 (+ 1129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:57.444461 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:57.444475 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:57.445006 (+ 531us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:31:57.445008 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:57.445016 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:31:57.445037 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:57.445418 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:57.445421 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:57.445425 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:31:57.445935 (+ 510us) client_negotiation.cc:240] Negotiation successful
0808 22:31:57.445970 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":70,"threads_started":1}
I20260808 22:31:57.449193 7361 tablet_service.cc:1467] Processing CreateTablet for tablet 4c3c1922e0a44a19b6e800b092fe4336 (DEFAULT_TABLE table=test-table [id=f7f853dd8d094d45a11a43adfe11db9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:31:57.449194 7742 tablet_service.cc:1467] Processing CreateTablet for tablet 4c3c1922e0a44a19b6e800b092fe4336 (DEFAULT_TABLE table=test-table [id=f7f853dd8d094d45a11a43adfe11db9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:31:57.449195 7554 tablet_service.cc:1467] Processing CreateTablet for tablet 4c3c1922e0a44a19b6e800b092fe4336 (DEFAULT_TABLE table=test-table [id=f7f853dd8d094d45a11a43adfe11db9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:31:57.450155 7361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c3c1922e0a44a19b6e800b092fe4336. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:57.450178 7742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c3c1922e0a44a19b6e800b092fe4336. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:57.450179 7554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c3c1922e0a44a19b6e800b092fe4336. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:57.456224 7866 tablet_bootstrap.cc:492] T 4c3c1922e0a44a19b6e800b092fe4336 P 9269f9cb31504d418d330020b13c816a: Bootstrap starting.
I20260808 22:31:57.457381 7867 tablet_bootstrap.cc:492] T 4c3c1922e0a44a19b6e800b092fe4336 P 2cfbe7cfb2e54457b49d48363aa69c91: Bootstrap starting.
I20260808 22:31:57.458336 7868 tablet_bootstrap.cc:492] T 4c3c1922e0a44a19b6e800b092fe4336 P a8ae993042fe4a16ae5f6d958b9f8eab: Bootstrap starting.
I20260808 22:31:57.459247 7866 tablet_bootstrap.cc:654] T 4c3c1922e0a44a19b6e800b092fe4336 P 9269f9cb31504d418d330020b13c816a: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:57.459414 7867 tablet_bootstrap.cc:654] T 4c3c1922e0a44a19b6e800b092fe4336 P 2cfbe7cfb2e54457b49d48363aa69c91: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:57.459977 7866 log.cc:824] T 4c3c1922e0a44a19b6e800b092fe4336 P 9269f9cb31504d418d330020b13c816a: Log is configured to *not* fsync() on all Append() calls
I20260808 22:31:57.460182 7868 tablet_bootstrap.cc:654] T 4c3c1922e0a44a19b6e800b092fe4336 P a8ae993042fe4a16ae5f6d958b9f8eab: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:57.460253 7867 log.cc:824] T 4c3c1922e0a44a19b6e800b092fe4336 P 2cfbe7cfb2e54457b49d48363aa69c91: Log is configured to *not* fsync() on all Append() calls
I20260808 22:31:57.460947 7868 log.cc:824] T 4c3c1922e0a44a19b6e800b092fe4336 P a8ae993042fe4a16ae5f6d958b9f8eab: Log is configured to *not* fsync() on all Append() calls
I20260808 22:31:57.461881 7866 tablet_bootstrap.cc:492] T 4c3c1922e0a44a19b6e800b092fe4336 P 9269f9cb31504d418d330020b13c816a: No bootstrap required, opened a new log
I20260808 22:31:57.462052 7866 ts_tablet_manager.cc:1397] T 4c3c1922e0a44a19b6e800b092fe4336 P 9269f9cb31504d418d330020b13c816a: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 22:31:57.462252 7867 tablet_bootstrap.cc:492] T 4c3c1922e0a44a19b6e800b092fe4336 P 2cfbe7cfb2e54457b49d48363aa69c91: No bootstrap required, opened a new log
I20260808 22:31:57.462443 7867 ts_tablet_manager.cc:1397] T 4c3c1922e0a44a19b6e800b092fe4336 P 2cfbe7cfb2e54457b49d48363aa69c91: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 22:31:57.463498 7868 tablet_bootstrap.cc:492] T 4c3c1922e0a44a19b6e800b092fe4336 P a8ae993042fe4a16ae5f6d958b9f8eab: No bootstrap required, opened a new log
I20260808 22:31:57.463675 7868 ts_tablet_manager.cc:1397] T 4c3c1922e0a44a19b6e800b092fe4336 P a8ae993042fe4a16ae5f6d958b9f8eab: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260808 22:31:57.465350 7866 raft_consensus.cc:348] T 4c3c1922e0a44a19b6e800b092fe4336 P 9269f9cb31504d418d330020b13c816a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9269f9cb31504d418d330020b13c816a" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 37307 } } peers { permanent_uuid: "2cfbe7cfb2e54457b49d48363aa69c91" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 33965 } } peers { permanent_uuid: "a8ae993042fe4a16ae5f6d958b9f8eab" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 43887 } }
I20260808 22:31:57.465557 7866 raft_consensus.cc:374] T 4c3c1922e0a44a19b6e800b092fe4336 P 9269f9cb31504d418d330020b13c816a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:57.465605 7866 raft_consensus.cc:729] T 4c3c1922e0a44a19b6e800b092fe4336 P 9269f9cb31504d418d330020b13c816a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9269f9cb31504d418d330020b13c816a, State: Initialized, Role: FOLLOWER
I20260808 22:31:57.466009 7866 consensus_queue.cc:260] T 4c3c1922e0a44a19b6e800b092fe4336 P 9269f9cb31504d418d330020b13c816a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9269f9cb31504d418d330020b13c816a" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 37307 } } peers { permanent_uuid: "2cfbe7cfb2e54457b49d48363aa69c91" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 33965 } } peers { permanent_uuid: "a8ae993042fe4a16ae5f6d958b9f8eab" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 43887 } }
I20260808 22:31:57.466585 7455 heartbeater.cc:503] Master 127.31.87.190:37163 was elected leader, sending a full tablet report...
I20260808 22:31:57.466931 7866 ts_tablet_manager.cc:1428] T 4c3c1922e0a44a19b6e800b092fe4336 P 9269f9cb31504d418d330020b13c816a: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 22:31:57.466837 7867 raft_consensus.cc:348] T 4c3c1922e0a44a19b6e800b092fe4336 P 2cfbe7cfb2e54457b49d48363aa69c91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9269f9cb31504d418d330020b13c816a" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 37307 } } peers { permanent_uuid: "2cfbe7cfb2e54457b49d48363aa69c91" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 33965 } } peers { permanent_uuid: "a8ae993042fe4a16ae5f6d958b9f8eab" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 43887 } }
I20260808 22:31:57.467077 7867 raft_consensus.cc:374] T 4c3c1922e0a44a19b6e800b092fe4336 P 2cfbe7cfb2e54457b49d48363aa69c91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:57.467139 7867 raft_consensus.cc:729] T 4c3c1922e0a44a19b6e800b092fe4336 P 2cfbe7cfb2e54457b49d48363aa69c91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cfbe7cfb2e54457b49d48363aa69c91, State: Initialized, Role: FOLLOWER
I20260808 22:31:57.467538 7867 consensus_queue.cc:260] T 4c3c1922e0a44a19b6e800b092fe4336 P 2cfbe7cfb2e54457b49d48363aa69c91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9269f9cb31504d418d330020b13c816a" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 37307 } } peers { permanent_uuid: "2cfbe7cfb2e54457b49d48363aa69c91" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 33965 } } peers { permanent_uuid: "a8ae993042fe4a16ae5f6d958b9f8eab" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 43887 } }
I20260808 22:31:57.468185 7648 heartbeater.cc:503] Master 127.31.87.190:37163 was elected leader, sending a full tablet report...
I20260808 22:31:57.468464 7867 ts_tablet_manager.cc:1428] T 4c3c1922e0a44a19b6e800b092fe4336 P 2cfbe7cfb2e54457b49d48363aa69c91: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 22:31:57.468307 7868 raft_consensus.cc:348] T 4c3c1922e0a44a19b6e800b092fe4336 P a8ae993042fe4a16ae5f6d958b9f8eab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9269f9cb31504d418d330020b13c816a" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 37307 } } peers { permanent_uuid: "2cfbe7cfb2e54457b49d48363aa69c91" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 33965 } } peers { permanent_uuid: "a8ae993042fe4a16ae5f6d958b9f8eab" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 43887 } }
I20260808 22:31:57.468533 7868 raft_consensus.cc:374] T 4c3c1922e0a44a19b6e800b092fe4336 P a8ae993042fe4a16ae5f6d958b9f8eab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:57.468576 7868 raft_consensus.cc:729] T 4c3c1922e0a44a19b6e800b092fe4336 P a8ae993042fe4a16ae5f6d958b9f8eab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8ae993042fe4a16ae5f6d958b9f8eab, State: Initialized, Role: FOLLOWER
I20260808 22:31:57.468971 7868 consensus_queue.cc:260] T 4c3c1922e0a44a19b6e800b092fe4336 P a8ae993042fe4a16ae5f6d958b9f8eab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9269f9cb31504d418d330020b13c816a" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 37307 } } peers { permanent_uuid: "2cfbe7cfb2e54457b49d48363aa69c91" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 33965 } } peers { permanent_uuid: "a8ae993042fe4a16ae5f6d958b9f8eab" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 43887 } }
I20260808 22:31:57.469607 7839 heartbeater.cc:503] Master 127.31.87.190:37163 was elected leader, sending a full tablet report...
I20260808 22:31:57.469905 7868 ts_tablet_manager.cc:1428] T 4c3c1922e0a44a19b6e800b092fe4336 P a8ae993042fe4a16ae5f6d958b9f8eab: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
W20260808 22:31:57.498950 7649 tablet.cc:2369] T 4c3c1922e0a44a19b6e800b092fe4336 P 2cfbe7cfb2e54457b49d48363aa69c91: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 22:31:57.578492 7456 tablet.cc:2369] T 4c3c1922e0a44a19b6e800b092fe4336 P 9269f9cb31504d418d330020b13c816a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:31:57.629578 7876 raft_consensus.cc:482] T 4c3c1922e0a44a19b6e800b092fe4336 P 2cfbe7cfb2e54457b49d48363aa69c91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:31:57.629812 7876 raft_consensus.cc:504] T 4c3c1922e0a44a19b6e800b092fe4336 P 2cfbe7cfb2e54457b49d48363aa69c91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9269f9cb31504d418d330020b13c816a" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 37307 } } peers { permanent_uuid: "2cfbe7cfb2e54457b49d48363aa69c91" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 33965 } } peers { permanent_uuid: "a8ae993042fe4a16ae5f6d958b9f8eab" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 43887 } }
I20260808 22:31:57.631141 7876 leader_election.cc:288] T 4c3c1922e0a44a19b6e800b092fe4336 P 2cfbe7cfb2e54457b49d48363aa69c91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9269f9cb31504d418d330020b13c816a (127.31.87.129:37307), a8ae993042fe4a16ae5f6d958b9f8eab (127.31.87.131:43887)
I20260808 22:31:57.632827 7650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:57.631353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:37307 (local address 127.31.87.130:40249)
0808 22:31:57.631473 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:57.631488 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:57.631563 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:57.631819 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:57.631822 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:57.631835 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:57.632320 (+ 485us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:31:57.632323 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:57.632335 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:31:57.632366 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:57.632598 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:57.632600 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:57.632603 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 22:31:57.632678 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 22:31:57.632710 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 22:31:57.632859 7862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:57.631435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:40249 (local address 127.31.87.129:37307)
0808 22:31:57.631553 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:57.631558 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:57.631575 (+ 17us) server_negotiation.cc:407] Connection header received
0808 22:31:57.631653 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:57.631657 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:57.631698 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:57.631782 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:57.632483 (+ 701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:57.632486 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:57.632488 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:31:57.632502 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:57.632522 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:31:57.632530 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:57.632607 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:57.632693 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 22:31:57.632737 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 22:31:57.633294 7864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:57.631487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:38389 (local address 127.31.87.131:43887)
0808 22:31:57.631601 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:57.631605 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:57.631877 (+ 272us) server_negotiation.cc:407] Connection header received
0808 22:31:57.631957 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:57.631959 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:57.631991 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:57.632063 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:57.632850 (+ 787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:57.632853 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:57.632856 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:31:57.632869 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:57.632892 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:31:57.632903 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:57.632985 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:57.633132 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 22:31:57.633176 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 22:31:57.633315 7883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:57.631370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.131:43887 (local address 127.31.87.130:38389)
0808 22:31:57.631783 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:57.631796 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:57.631878 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:57.632105 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:57.632108 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:57.632124 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:57.632666 (+ 542us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:31:57.632669 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:57.632679 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:31:57.632708 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:57.633029 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:57.633032 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:57.633036 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:31:57.633136 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 22:31:57.633176 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":102,"thread_start_us":125,"threads_started":1}
I20260808 22:31:57.633455 7392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c3c1922e0a44a19b6e800b092fe4336" candidate_uuid: "2cfbe7cfb2e54457b49d48363aa69c91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9269f9cb31504d418d330020b13c816a" is_pre_election: true
I20260808 22:31:57.633714 7392 raft_consensus.cc:2454] T 4c3c1922e0a44a19b6e800b092fe4336 P 9269f9cb31504d418d330020b13c816a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2cfbe7cfb2e54457b49d48363aa69c91 in term 0.
I20260808 22:31:57.634069 7768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c3c1922e0a44a19b6e800b092fe4336" candidate_uuid: "2cfbe7cfb2e54457b49d48363aa69c91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ae993042fe4a16ae5f6d958b9f8eab" is_pre_election: true
I20260808 22:31:57.634196 7482 leader_election.cc:302] T 4c3c1922e0a44a19b6e800b092fe4336 P 2cfbe7cfb2e54457b49d48363aa69c91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cfbe7cfb2e54457b49d48363aa69c91, 9269f9cb31504d418d330020b13c816a; no voters:
I20260808 22:31:57.634351 7768 raft_consensus.cc:2454] T 4c3c1922e0a44a19b6e800b092fe4336 P a8ae993042fe4a16ae5f6d958b9f8eab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2cfbe7cfb2e54457b49d48363aa69c91 in term 0.
I20260808 22:31:57.634476 7876 raft_consensus.cc:2781] T 4c3c1922e0a44a19b6e800b092fe4336 P 2cfbe7cfb2e54457b49d48363aa69c91 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:31:57.634569 7876 raft_consensus.cc:482] T 4c3c1922e0a44a19b6e800b092fe4336 P 2cfbe7cfb2e54457b49d48363aa69c91 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:31:57.634614 7876 raft_consensus.cc:3037] T 4c3c1922e0a44a19b6e800b092fe4336 P 2cfbe7cfb2e54457b49d48363aa69c91 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:57.635624 7876 raft_consensus.cc:504] T 4c3c1922e0a44a19b6e800b092fe4336 P 2cfbe7cfb2e54457b49d48363aa69c91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9269f9cb31504d418d330020b13c816a" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 37307 } } peers { permanent_uuid: "2cfbe7cfb2e54457b49d48363aa69c91" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 33965 } } peers { permanent_uuid: "a8ae993042fe4a16ae5f6d958b9f8eab" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 43887 } }
I20260808 22:31:57.635924 7876 leader_election.cc:288] T 4c3c1922e0a44a19b6e800b092fe4336 P 2cfbe7cfb2e54457b49d48363aa69c91 [CANDIDATE]: Term 1 election: Requested vote from peers 9269f9cb31504d418d330020b13c816a (127.31.87.129:37307), a8ae993042fe4a16ae5f6d958b9f8eab (127.31.87.131:43887)
I20260808 22:31:57.636232 7392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c3c1922e0a44a19b6e800b092fe4336" candidate_uuid: "2cfbe7cfb2e54457b49d48363aa69c91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9269f9cb31504d418d330020b13c816a"
I20260808 22:31:57.636354 7392 raft_consensus.cc:3037] T 4c3c1922e0a44a19b6e800b092fe4336 P 9269f9cb31504d418d330020b13c816a [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:57.636375 7768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c3c1922e0a44a19b6e800b092fe4336" candidate_uuid: "2cfbe7cfb2e54457b49d48363aa69c91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ae993042fe4a16ae5f6d958b9f8eab"
I20260808 22:31:57.636480 7768 raft_consensus.cc:3037] T 4c3c1922e0a44a19b6e800b092fe4336 P a8ae993042fe4a16ae5f6d958b9f8eab [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:57.637470 7392 raft_consensus.cc:2454] T 4c3c1922e0a44a19b6e800b092fe4336 P 9269f9cb31504d418d330020b13c816a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2cfbe7cfb2e54457b49d48363aa69c91 in term 1.
I20260808 22:31:57.637511 7768 raft_consensus.cc:2454] T 4c3c1922e0a44a19b6e800b092fe4336 P a8ae993042fe4a16ae5f6d958b9f8eab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2cfbe7cfb2e54457b49d48363aa69c91 in term 1.
I20260808 22:31:57.637722 7482 leader_election.cc:302] T 4c3c1922e0a44a19b6e800b092fe4336 P 2cfbe7cfb2e54457b49d48363aa69c91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cfbe7cfb2e54457b49d48363aa69c91, 9269f9cb31504d418d330020b13c816a; no voters:
I20260808 22:31:57.637945 7876 raft_consensus.cc:2781] T 4c3c1922e0a44a19b6e800b092fe4336 P 2cfbe7cfb2e54457b49d48363aa69c91 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:31:57.638156 7876 raft_consensus.cc:686] T 4c3c1922e0a44a19b6e800b092fe4336 P 2cfbe7cfb2e54457b49d48363aa69c91 [term 1 LEADER]: Becoming Leader. State: Replica: 2cfbe7cfb2e54457b49d48363aa69c91, State: Running, Role: LEADER
I20260808 22:31:57.638434 7876 consensus_queue.cc:237] T 4c3c1922e0a44a19b6e800b092fe4336 P 2cfbe7cfb2e54457b49d48363aa69c91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9269f9cb31504d418d330020b13c816a" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 37307 } } peers { permanent_uuid: "2cfbe7cfb2e54457b49d48363aa69c91" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 33965 } } peers { permanent_uuid: "a8ae993042fe4a16ae5f6d958b9f8eab" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 43887 } }
I20260808 22:31:57.641367 7171 catalog_manager.cc:5495] T 4c3c1922e0a44a19b6e800b092fe4336 P 2cfbe7cfb2e54457b49d48363aa69c91 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2cfbe7cfb2e54457b49d48363aa69c91 (127.31.87.130). New cstate: current_term: 1 leader_uuid: "2cfbe7cfb2e54457b49d48363aa69c91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9269f9cb31504d418d330020b13c816a" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 37307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2cfbe7cfb2e54457b49d48363aa69c91" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 33965 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a8ae993042fe4a16ae5f6d958b9f8eab" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 43887 } health_report { overall_health: UNKNOWN } } }
W20260808 22:31:57.667968 7840 tablet.cc:2369] T 4c3c1922e0a44a19b6e800b092fe4336 P a8ae993042fe4a16ae5f6d958b9f8eab: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:31:57.682322 7865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:57.680889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50606 (local address 127.31.87.130:33965)
0808 22:31:57.681025 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:57.681030 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:57.681044 (+ 14us) server_negotiation.cc:407] Connection header received
0808 22:31:57.681149 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:57.681152 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:57.681216 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:57.681289 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:57.681953 (+ 664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:57.681957 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:57.681960 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:31:57.681972 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:57.681993 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:31:57.682001 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:57.682073 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:57.682154 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 22:31:57.682196 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 22:31:57.688031 7392 raft_consensus.cc:1261] T 4c3c1922e0a44a19b6e800b092fe4336 P 9269f9cb31504d418d330020b13c816a [term 1 FOLLOWER]: Refusing update from remote peer 2cfbe7cfb2e54457b49d48363aa69c91: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:31:57.688031 7768 raft_consensus.cc:1261] T 4c3c1922e0a44a19b6e800b092fe4336 P a8ae993042fe4a16ae5f6d958b9f8eab [term 1 FOLLOWER]: Refusing update from remote peer 2cfbe7cfb2e54457b49d48363aa69c91: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:31:57.688755 7884 consensus_queue.cc:1035] T 4c3c1922e0a44a19b6e800b092fe4336 P 2cfbe7cfb2e54457b49d48363aa69c91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ae993042fe4a16ae5f6d958b9f8eab" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:31:57.688935 7876 consensus_queue.cc:1035] T 4c3c1922e0a44a19b6e800b092fe4336 P 2cfbe7cfb2e54457b49d48363aa69c91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9269f9cb31504d418d330020b13c816a" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 37307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:31:57.698570 7889 mvcc.cc:205] Tried to move back new op lower bound from 7316391189243224064 to 7316391189049413632. Current Snapshot: MvccSnapshot[applied={T|T < 7316391189243224064 or (T in {7316391189243224064})}]
I20260808 22:31:57.706023 7169 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35194:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:31:57.706182 7169 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35194:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:31:57.708573 7169 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 28bbee1727ee4adda5a3cffccc322c73: Sending DeleteTablet for 3 replicas of tablet 4c3c1922e0a44a19b6e800b092fe4336
I20260808 22:31:57.709301 7361 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c3c1922e0a44a19b6e800b092fe4336 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:31:57 UTC) from {username='slave'} at 127.0.0.1:41420
I20260808 22:31:57.709381 7742 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c3c1922e0a44a19b6e800b092fe4336 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:31:57 UTC) from {username='slave'} at 127.0.0.1:38792
I20260808 22:31:57.709367 7554 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c3c1922e0a44a19b6e800b092fe4336 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:31:57 UTC) from {username='slave'} at 127.0.0.1:50590
I20260808 22:31:57.709921 7904 tablet_replica.cc:330] T 4c3c1922e0a44a19b6e800b092fe4336 P 9269f9cb31504d418d330020b13c816a: stopping tablet replica
I20260808 22:31:57.709919 7907 tablet_replica.cc:330] T 4c3c1922e0a44a19b6e800b092fe4336 P a8ae993042fe4a16ae5f6d958b9f8eab: stopping tablet replica
I20260808 22:31:57.710216 7907 raft_consensus.cc:2229] T 4c3c1922e0a44a19b6e800b092fe4336 P a8ae993042fe4a16ae5f6d958b9f8eab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:31:57.710216 7904 raft_consensus.cc:2229] T 4c3c1922e0a44a19b6e800b092fe4336 P 9269f9cb31504d418d330020b13c816a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:31:57.710521 7907 raft_consensus.cc:2258] T 4c3c1922e0a44a19b6e800b092fe4336 P a8ae993042fe4a16ae5f6d958b9f8eab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:31:57.710531 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 7262
I20260808 22:31:57.710994 7908 tablet_replica.cc:330] T 4c3c1922e0a44a19b6e800b092fe4336 P 2cfbe7cfb2e54457b49d48363aa69c91: stopping tablet replica
I20260808 22:31:57.711232 7908 raft_consensus.cc:2229] T 4c3c1922e0a44a19b6e800b092fe4336 P 2cfbe7cfb2e54457b49d48363aa69c91 [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:31:57.711535 7908 raft_consensus.cc:2258] T 4c3c1922e0a44a19b6e800b092fe4336 P 2cfbe7cfb2e54457b49d48363aa69c91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:31:57.725363 7908 ts_tablet_manager.cc:1905] T 4c3c1922e0a44a19b6e800b092fe4336 P 2cfbe7cfb2e54457b49d48363aa69c91: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:31:57.727829 7908 ts_tablet_manager.cc:1918] T 4c3c1922e0a44a19b6e800b092fe4336 P 2cfbe7cfb2e54457b49d48363aa69c91: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:31:57.727972 7908 log.cc:1196] T 4c3c1922e0a44a19b6e800b092fe4336 P 2cfbe7cfb2e54457b49d48363aa69c91: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/ts-1/wal/wals/4c3c1922e0a44a19b6e800b092fe4336
I20260808 22:31:57.728299 7908 ts_tablet_manager.cc:1939] T 4c3c1922e0a44a19b6e800b092fe4336 P 2cfbe7cfb2e54457b49d48363aa69c91: Deleting consensus metadata
I20260808 22:31:57.729403 7153 catalog_manager.cc:4841] TS 2cfbe7cfb2e54457b49d48363aa69c91 (127.31.87.130:33965): tablet 4c3c1922e0a44a19b6e800b092fe4336 (table test-table [id=f7f853dd8d094d45a11a43adfe11db9b]) successfully deleted
I20260808 22:31:57.736419 7907 ts_tablet_manager.cc:1905] T 4c3c1922e0a44a19b6e800b092fe4336 P a8ae993042fe4a16ae5f6d958b9f8eab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:31:57.739291 7907 ts_tablet_manager.cc:1918] T 4c3c1922e0a44a19b6e800b092fe4336 P a8ae993042fe4a16ae5f6d958b9f8eab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:31:57.739400 7907 log.cc:1196] T 4c3c1922e0a44a19b6e800b092fe4336 P a8ae993042fe4a16ae5f6d958b9f8eab: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786228291013016-32094-0/minicluster-data/ts-2/wal/wals/4c3c1922e0a44a19b6e800b092fe4336
I20260808 22:31:57.739753 7907 ts_tablet_manager.cc:1939] T 4c3c1922e0a44a19b6e800b092fe4336 P a8ae993042fe4a16ae5f6d958b9f8eab: Deleting consensus metadata
W20260808 22:31:57.740475 7152 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.87.129:37307 (error 108)
I20260808 22:31:57.740863 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 7458
I20260808 22:31:57.741539 7858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:57.741229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:37307 (local address 127.0.0.1:41422)
0808 22:31:57.741388 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:57.741454 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.129:37307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
W20260808 22:31:57.741724 7152 catalog_manager.cc:4584] TS 9269f9cb31504d418d330020b13c816a (127.31.87.129:37307): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4c3c1922e0a44a19b6e800b092fe4336: Network error: Client connection negotiation failed: client connection to 127.31.87.129:37307: connect: Connection refused (error 111)
I20260808 22:31:57.745239 7151 catalog_manager.cc:4841] TS a8ae993042fe4a16ae5f6d958b9f8eab (127.31.87.131:43887): tablet 4c3c1922e0a44a19b6e800b092fe4336 (table test-table [id=f7f853dd8d094d45a11a43adfe11db9b]) successfully deleted
I20260808 22:31:57.752678 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 7651
I20260808 22:31:57.760805 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 7134
2026-08-08T22:31:57Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1075 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260808 22:31:57.783627 32094 mini_oidc.cc:152] Starting JWKS server
I20260808 22:31:57.785298 32094 webserver.cc:460] Webserver started at https://127.0.0.1:39465/ using document root <none> and password file <none>
I20260808 22:31:57.785458 32094 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 22:31:57.785714 32094 webserver.cc:460] Webserver started at http://127.0.0.1:43221/ using document root <none> and password file <none>
2026-08-08T22:31:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:31:57Z Disabled control of system clock
I20260808 22:31:57.794984 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:40919
--webserver_interface=127.31.87.190
--webserver_port=0
--builtin_ntp_servers=127.31.87.148:33321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:40919
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39465/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786228291013016-32094-0/testchainca.cert with env {}
W20260808 22:31:57.906320 7915 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:57.906553 7915 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:57.906589 7915 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:57.910457 7915 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:31:57.910502 7915 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:57.910517 7915 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:57.910540 7915 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:31:57.910558 7915 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 22:31:57.910578 7915 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 22:31:57.910597 7915 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39465/jwks.json
I20260808 22:31:57.915804 7915 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:33321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786228291013016-32094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:40919
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:40919
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39465/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.87.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786228291013016-32094-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7915
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:57.916715 7915 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:57.917637 7915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:57.925578 7923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:57.925738 7915 server_base.cc:1034] running on GCE node
W20260808 22:31:57.925566 7925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:57.925601 7921 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:57.926322 7915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:57.927280 7915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:57.928435 7915 hybrid_clock.cc:648] HybridClock initialized: now 1786228317928417 us; error 28 us; skew 500 ppm
I20260808 22:31:57.930115 7915 webserver.cc:460] Webserver started at http://127.31.87.190:41075/ using document root <none> and password file <none>
I20260808 22:31:57.930608 7915 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:57.930663 7915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:57.930819 7915 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:57.932623 7915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786228291013016-32094-0/minicluster-data/master-0/data/instance:
uuid: "3d17852405a745d49642a94643cac3da"
format_stamp: "Formatted at 2026-08-08 22:31:57 on dist-test-slave-266d"
server_key: "df2794d45c69b2a3eed286e25561d3fe"
server_key_iv: "350fe80f85d7a35aa0ba9e948edf6b0a"
server_key_version: "encryptionkey@0"
I20260808 22:31:57.933123 7915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786228291013016-32094-0/minicluster-data/master-0/wal/instance:
uuid: "3d17852405a745d49642a94643cac3da"
format_stamp: "Formatted at 2026-08-08 22:31:57 on dist-test-slave-266d"
server_key: "df2794d45c69b2a3eed286e25561d3fe"
server_key_iv: "350fe80f85d7a35aa0ba9e948edf6b0a"
server_key_version: "encryptionkey@0"
I20260808 22:31:57.936446 7915 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.002s
I20260808 22:31:57.938653 7934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:57.939786 7915 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:31:57.939879 7915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "3d17852405a745d49642a94643cac3da"
format_stamp: "Formatted at 2026-08-08 22:31:57 on dist-test-slave-266d"
server_key: "df2794d45c69b2a3eed286e25561d3fe"
server_key_iv: "350fe80f85d7a35aa0ba9e948edf6b0a"
server_key_version: "encryptionkey@0"
I20260808 22:31:57.939960 7915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:57.966183 7915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:57.966774 7915 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:57.966913 7915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:57.973390 7915 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.190:40919
I20260808 22:31:57.973404 8019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.190:40919 every 8 connection(s)
I20260808 22:31:57.974480 7915 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:31:57.977432 8021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:57.980391 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 7915
I20260808 22:31:57.980513 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786228291013016-32094-0/minicluster-data/master-0/wal/instance
I20260808 22:31:57.980804 32094 external_mini_cluster.cc:1428] Setting key f50dbefe76439889c4f8acc87f4bf9d4
I20260808 22:31:57.983165 8021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d17852405a745d49642a94643cac3da: Bootstrap starting.
I20260808 22:31:57.985245 8021 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d17852405a745d49642a94643cac3da: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:57.986011 8021 log.cc:824] T 00000000000000000000000000000000 P 3d17852405a745d49642a94643cac3da: Log is configured to *not* fsync() on all Append() calls
I20260808 22:31:57.987800 8021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d17852405a745d49642a94643cac3da: No bootstrap required, opened a new log
I20260808 22:31:57.989516 8026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:57.982181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51292 (local address 127.31.87.190:40919)
0808 22:31:57.982801 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:57.982816 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:57.982849 (+ 33us) server_negotiation.cc:407] Connection header received
0808 22:31:57.983488 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:57.983507 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:57.983886 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:57.984176 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:57.984593 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:57.985627 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:57.986542 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:57.986999 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:57.987141 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:57.987929 (+ 788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:57.987934 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:57.987946 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:31:57.988300 (+ 354us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:57.988357 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:31:57.988376 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:57.988461 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:57.988608 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 22:31:57.988838 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":118,"threads_started":1}
I20260808 22:31:57.991381 8021 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d17852405a745d49642a94643cac3da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d17852405a745d49642a94643cac3da" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 40919 } }
I20260808 22:31:57.991603 8021 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d17852405a745d49642a94643cac3da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:57.991662 8021 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d17852405a745d49642a94643cac3da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d17852405a745d49642a94643cac3da, State: Initialized, Role: FOLLOWER
I20260808 22:31:57.992080 8021 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d17852405a745d49642a94643cac3da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d17852405a745d49642a94643cac3da" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 40919 } }
I20260808 22:31:57.992223 8021 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d17852405a745d49642a94643cac3da [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:31:57.992230 32094 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:31:57.992291 8021 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d17852405a745d49642a94643cac3da [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:31:57.992379 8021 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d17852405a745d49642a94643cac3da [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:57.993338 8021 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d17852405a745d49642a94643cac3da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d17852405a745d49642a94643cac3da" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 40919 } }
I20260808 22:31:57.993667 8021 leader_election.cc:302] T 00000000000000000000000000000000 P 3d17852405a745d49642a94643cac3da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d17852405a745d49642a94643cac3da; no voters:
I20260808 22:31:57.993988 8021 leader_election.cc:288] T 00000000000000000000000000000000 P 3d17852405a745d49642a94643cac3da [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:31:57.994066 8029 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d17852405a745d49642a94643cac3da [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:31:57.994278 8029 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d17852405a745d49642a94643cac3da [term 1 LEADER]: Becoming Leader. State: Replica: 3d17852405a745d49642a94643cac3da, State: Running, Role: LEADER
I20260808 22:31:57.994562 8029 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d17852405a745d49642a94643cac3da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d17852405a745d49642a94643cac3da" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 40919 } }
I20260808 22:31:57.995072 8021 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d17852405a745d49642a94643cac3da [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:31:57.995970 8033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d17852405a745d49642a94643cac3da [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d17852405a745d49642a94643cac3da. Latest consensus state: current_term: 1 leader_uuid: "3d17852405a745d49642a94643cac3da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d17852405a745d49642a94643cac3da" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 40919 } } }
I20260808 22:31:57.996096 8033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d17852405a745d49642a94643cac3da [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:57.996372 8031 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d17852405a745d49642a94643cac3da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d17852405a745d49642a94643cac3da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d17852405a745d49642a94643cac3da" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 40919 } } }
I20260808 22:31:57.996476 8031 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d17852405a745d49642a94643cac3da [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:57.996522 8040 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:31:57.999500 8040 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:31:58.004894 8040 catalog_manager.cc:1347] Generated new cluster ID: e061c7dc086f4096bcfd6b862116e1e4
I20260808 22:31:58.004956 8040 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:31:58.010726 8040 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:31:58.011525 8040 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:31:58.017120 8040 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d17852405a745d49642a94643cac3da: Generated new TSK 0
I20260808 22:31:58.017683 8040 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 22:31:58.055446 8063 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260808 22:31:58.055599 8063 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 22:31:58.056082 8026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:58.032843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51308 (local address 127.31.87.190:40919)
0808 22:31:58.032969 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:58.032974 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:58.033196 (+ 222us) server_negotiation.cc:407] Connection header received
0808 22:31:58.033409 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:58.033411 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:58.033456 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:58.033533 (+ 77us) server_negotiation.cc:226] Negotiated authn=JWT
0808 22:31:58.033797 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:58.034320 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:58.035498 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:58.035725 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:58.035805 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:58.036120 (+ 315us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 22:31:58.055736 (+ 19616us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39465/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 22:31:58.055970 (+ 234us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51308: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39465/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":47}
W20260808 22:31:58.056154 8026 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51308: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39465/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260808 22:31:58.057741 8062 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:31:58.032762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:40919 (local address 127.0.0.1:51308)
0808 22:31:58.033125 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:58.033141 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:58.033272 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:58.033564 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:58.033567 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:58.033576 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0808 22:31:58.033704 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:58.033710 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:58.034421 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:58.034425 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:58.035400 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:58.035409 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:58.035821 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:58.035824 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:58.035918 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:58.035971 (+ 53us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 22:31:58.055969 (+ 19998us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39465/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 22:31:58.056220 (+ 251us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.31.87.190:40919: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39465/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":88,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.31.87.190:40919: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39465/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260808 22:31:58.058493 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 7915
2026-08-08T22:31:58Z chronyd exiting
I20260808 22:31:58.102963 32094 test_util.cc:175] -----------------------------------------------
I20260808 22:31:58.103075 32094 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786228291013016-32094-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (321 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260808 22:31:58.104568 32094 mini_oidc.cc:152] Starting JWKS server
I20260808 22:31:58.105806 32094 webserver.cc:460] Webserver started at https://127.0.0.1:35767/ using document root <none> and password file <none>
I20260808 22:31:58.105922 32094 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 22:31:58.106208 32094 webserver.cc:460] Webserver started at http://127.0.0.1:37959/ using document root <none> and password file <none>
2026-08-08T22:31:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:31:58Z Disabled control of system clock
I20260808 22:31:58.122784 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:39531
--webserver_interface=127.31.87.190
--webserver_port=0
--builtin_ntp_servers=127.31.87.148:41721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:39531
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35767/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786228291013016-32094-0/testchainca.cert with env {}
W20260808 22:31:58.232962 8070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:58.233214 8070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:58.233251 8070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:58.237211 8070 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:31:58.237259 8070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:58.237280 8070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:58.237296 8070 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:31:58.237309 8070 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 22:31:58.237330 8070 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 22:31:58.237342 8070 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35767/jwks.json
I20260808 22:31:58.242367 8070 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:41721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786228291013016-32094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:39531
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:39531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35767/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.87.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786228291013016-32094-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8070
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:58.243229 8070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:58.244107 8070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:58.251974 8080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:58.251962 8077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:58.251971 8082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:58.252051 8070 server_base.cc:1034] running on GCE node
I20260808 22:31:58.252759 8070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:58.253821 8070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:58.254981 8070 hybrid_clock.cc:648] HybridClock initialized: now 1786228318254957 us; error 36 us; skew 500 ppm
I20260808 22:31:58.256659 8070 webserver.cc:460] Webserver started at http://127.31.87.190:44357/ using document root <none> and password file <none>
I20260808 22:31:58.257223 8070 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:58.257294 8070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:58.257483 8070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:58.259357 8070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786228291013016-32094-0/minicluster-data/master-0/data/instance:
uuid: "32ca4e090e7141a0803c79102fffb61b"
format_stamp: "Formatted at 2026-08-08 22:31:58 on dist-test-slave-266d"
server_key: "07871a915fd6038e41520d124c722118"
server_key_iv: "d17d6b427819800f210edfd999a91604"
server_key_version: "encryptionkey@0"
I20260808 22:31:58.259881 8070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786228291013016-32094-0/minicluster-data/master-0/wal/instance:
uuid: "32ca4e090e7141a0803c79102fffb61b"
format_stamp: "Formatted at 2026-08-08 22:31:58 on dist-test-slave-266d"
server_key: "07871a915fd6038e41520d124c722118"
server_key_iv: "d17d6b427819800f210edfd999a91604"
server_key_version: "encryptionkey@0"
I20260808 22:31:58.263252 8070 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 22:31:58.265480 8088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:58.266582 8070 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:31:58.266692 8070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "32ca4e090e7141a0803c79102fffb61b"
format_stamp: "Formatted at 2026-08-08 22:31:58 on dist-test-slave-266d"
server_key: "07871a915fd6038e41520d124c722118"
server_key_iv: "d17d6b427819800f210edfd999a91604"
server_key_version: "encryptionkey@0"
I20260808 22:31:58.266780 8070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:58.291882 8070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:58.292472 8070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:58.292646 8070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:58.299045 8070 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.190:39531
I20260808 22:31:58.299054 8163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.190:39531 every 8 connection(s)
I20260808 22:31:58.299908 8070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:31:58.303007 8166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:58.305341 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 8070
I20260808 22:31:58.305450 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786228291013016-32094-0/minicluster-data/master-0/wal/instance
I20260808 22:31:58.305707 32094 external_mini_cluster.cc:1428] Setting key 2dad30bb75fc29a46b78273866580b32
I20260808 22:31:58.308390 8166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32ca4e090e7141a0803c79102fffb61b: Bootstrap starting.
I20260808 22:31:58.310686 8166 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 32ca4e090e7141a0803c79102fffb61b: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:58.311486 8166 log.cc:824] T 00000000000000000000000000000000 P 32ca4e090e7141a0803c79102fffb61b: Log is configured to *not* fsync() on all Append() calls
I20260808 22:31:58.313175 8166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32ca4e090e7141a0803c79102fffb61b: No bootstrap required, opened a new log
I20260808 22:31:58.314275 8171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:58.306901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42688 (local address 127.31.87.190:39531)
0808 22:31:58.307475 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:58.307489 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:58.307523 (+ 34us) server_negotiation.cc:407] Connection header received
0808 22:31:58.308146 (+ 623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:58.308162 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:58.308546 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:58.308824 (+ 278us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:58.309193 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:58.310108 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:58.311264 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:58.311741 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:58.311896 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:58.312672 (+ 776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:58.312678 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:58.312690 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:31:58.313037 (+ 347us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:58.313125 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:31:58.313155 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:58.313241 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:58.313379 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 22:31:58.313620 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":118,"threads_started":1}
I20260808 22:31:58.316995 32094 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:31:58.317020 8166 raft_consensus.cc:348] T 00000000000000000000000000000000 P 32ca4e090e7141a0803c79102fffb61b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32ca4e090e7141a0803c79102fffb61b" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 39531 } }
I20260808 22:31:58.317269 8166 raft_consensus.cc:374] T 00000000000000000000000000000000 P 32ca4e090e7141a0803c79102fffb61b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:58.317337 8166 raft_consensus.cc:729] T 00000000000000000000000000000000 P 32ca4e090e7141a0803c79102fffb61b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32ca4e090e7141a0803c79102fffb61b, State: Initialized, Role: FOLLOWER
I20260808 22:31:58.317736 8166 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32ca4e090e7141a0803c79102fffb61b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32ca4e090e7141a0803c79102fffb61b" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 39531 } }
I20260808 22:31:58.317890 8166 raft_consensus.cc:388] T 00000000000000000000000000000000 P 32ca4e090e7141a0803c79102fffb61b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:31:58.317958 8166 raft_consensus.cc:482] T 00000000000000000000000000000000 P 32ca4e090e7141a0803c79102fffb61b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:31:58.318053 8166 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 32ca4e090e7141a0803c79102fffb61b [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:58.318967 8166 raft_consensus.cc:504] T 00000000000000000000000000000000 P 32ca4e090e7141a0803c79102fffb61b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32ca4e090e7141a0803c79102fffb61b" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 39531 } }
I20260808 22:31:58.319238 8166 leader_election.cc:302] T 00000000000000000000000000000000 P 32ca4e090e7141a0803c79102fffb61b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 32ca4e090e7141a0803c79102fffb61b; no voters:
I20260808 22:31:58.319470 8166 leader_election.cc:288] T 00000000000000000000000000000000 P 32ca4e090e7141a0803c79102fffb61b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:31:58.319600 8174 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 32ca4e090e7141a0803c79102fffb61b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:31:58.319808 8174 raft_consensus.cc:686] T 00000000000000000000000000000000 P 32ca4e090e7141a0803c79102fffb61b [term 1 LEADER]: Becoming Leader. State: Replica: 32ca4e090e7141a0803c79102fffb61b, State: Running, Role: LEADER
I20260808 22:31:58.320113 8174 consensus_queue.cc:237] T 00000000000000000000000000000000 P 32ca4e090e7141a0803c79102fffb61b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32ca4e090e7141a0803c79102fffb61b" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 39531 } }
I20260808 22:31:58.320504 8166 sys_catalog.cc:564] T 00000000000000000000000000000000 P 32ca4e090e7141a0803c79102fffb61b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:31:58.321637 8177 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32ca4e090e7141a0803c79102fffb61b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "32ca4e090e7141a0803c79102fffb61b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32ca4e090e7141a0803c79102fffb61b" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 39531 } } }
I20260808 22:31:58.321637 8179 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32ca4e090e7141a0803c79102fffb61b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 32ca4e090e7141a0803c79102fffb61b. Latest consensus state: current_term: 1 leader_uuid: "32ca4e090e7141a0803c79102fffb61b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32ca4e090e7141a0803c79102fffb61b" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 39531 } } }
I20260808 22:31:58.321780 8177 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32ca4e090e7141a0803c79102fffb61b [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:58.321802 8179 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32ca4e090e7141a0803c79102fffb61b [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:58.322149 8190 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:31:58.324819 8190 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:31:58.329742 8190 catalog_manager.cc:1347] Generated new cluster ID: 3e3ba06a62074674aaabe1dbea20d780
I20260808 22:31:58.329802 8190 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:31:58.338640 8190 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:31:58.339449 8190 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:31:58.346534 8190 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 32ca4e090e7141a0803c79102fffb61b: Generated new TSK 0
I20260808 22:31:58.347091 8190 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 22:31:58.370280 8214 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260808 22:31:58.370368 8214 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 22:31:58.370930 8171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:58.355805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42700 (local address 127.31.87.190:39531)
0808 22:31:58.355924 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:58.355930 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:58.356093 (+ 163us) server_negotiation.cc:407] Connection header received
0808 22:31:58.356242 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:58.356244 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:58.356286 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:58.356376 (+ 90us) server_negotiation.cc:226] Negotiated authn=JWT
0808 22:31:58.356596 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:58.357085 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:58.358344 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:58.358552 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:58.358619 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:58.358899 (+ 280us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 22:31:58.370631 (+ 11732us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35767/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 22:31:58.370825 (+ 194us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42700: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35767/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":45}
W20260808 22:31:58.370983 8171 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42700: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35767/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260808 22:31:58.371417 8210 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:31:58.355728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:39531 (local address 127.0.0.1:42700)
0808 22:31:58.356013 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:58.356027 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:58.356136 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:58.356392 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:58.356395 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:58.356403 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0808 22:31:58.356512 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:58.356518 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:58.357243 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:58.357251 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:58.358236 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:58.358246 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:58.358655 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:58.358659 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:58.358744 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:58.358772 (+ 28us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 22:31:58.370790 (+ 12018us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35767/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 22:31:58.370952 (+ 162us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.31.87.190:39531: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35767/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":82,"threads_started":1}
I20260808 22:31:58.371999 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 8070
2026-08-08T22:31:58Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (314 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260808 22:31:58.418444 32094 mini_oidc.cc:152] Starting JWKS server
I20260808 22:31:58.419275 32094 webserver.cc:460] Webserver started at https://127.0.0.1:44069/ using document root <none> and password file <none>
I20260808 22:31:58.419369 32094 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 22:31:58.419612 32094 webserver.cc:460] Webserver started at http://127.0.0.1:33731/ using document root <none> and password file <none>
2026-08-08T22:31:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:31:58Z Disabled control of system clock
I20260808 22:31:58.428640 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:44333
--webserver_interface=127.31.87.190
--webserver_port=0
--builtin_ntp_servers=127.31.87.148:41201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:44333
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44069/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 22:31:58.535123 8221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:58.535362 8221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:58.535399 8221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:58.539224 8221 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:31:58.539273 8221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:58.539290 8221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:58.539309 8221 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:31:58.539331 8221 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 22:31:58.539355 8221 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 22:31:58.539377 8221 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44069/jwks.json
I20260808 22:31:58.543903 8221 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:41201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786228291013016-32094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:44333
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:44333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44069/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.87.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:58.544797 8221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:58.546065 8221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:58.554015 8228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:58.554029 8227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:58.554018 8230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:58.554246 8221 server_base.cc:1034] running on GCE node
I20260808 22:31:58.554841 8221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:58.555841 8221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:58.556989 8221 hybrid_clock.cc:648] HybridClock initialized: now 1786228318556977 us; error 33 us; skew 500 ppm
I20260808 22:31:58.558708 8221 webserver.cc:460] Webserver started at http://127.31.87.190:41139/ using document root <none> and password file <none>
I20260808 22:31:58.559233 8221 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:58.559299 8221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:58.559478 8221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:58.561419 8221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786228291013016-32094-0/minicluster-data/master-0/data/instance:
uuid: "d0a5a49c7d18482ba56669b2fad3a41a"
format_stamp: "Formatted at 2026-08-08 22:31:58 on dist-test-slave-266d"
server_key: "078b86f4736ec93f833fcc35a0479884"
server_key_iv: "12dcebbf8fb1940e09df95b87ea6a6f3"
server_key_version: "encryptionkey@0"
I20260808 22:31:58.561924 8221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786228291013016-32094-0/minicluster-data/master-0/wal/instance:
uuid: "d0a5a49c7d18482ba56669b2fad3a41a"
format_stamp: "Formatted at 2026-08-08 22:31:58 on dist-test-slave-266d"
server_key: "078b86f4736ec93f833fcc35a0479884"
server_key_iv: "12dcebbf8fb1940e09df95b87ea6a6f3"
server_key_version: "encryptionkey@0"
I20260808 22:31:58.565316 8221 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:31:58.567502 8242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:58.568584 8221 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:31:58.568702 8221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "d0a5a49c7d18482ba56669b2fad3a41a"
format_stamp: "Formatted at 2026-08-08 22:31:58 on dist-test-slave-266d"
server_key: "078b86f4736ec93f833fcc35a0479884"
server_key_iv: "12dcebbf8fb1940e09df95b87ea6a6f3"
server_key_version: "encryptionkey@0"
I20260808 22:31:58.568789 8221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:58.591033 8221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:58.591642 8221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:58.591802 8221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:58.598240 8221 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.190:44333
I20260808 22:31:58.598254 8315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.190:44333 every 8 connection(s)
I20260808 22:31:58.599102 8221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:31:58.601696 8317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:58.603572 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 8221
I20260808 22:31:58.603688 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786228291013016-32094-0/minicluster-data/master-0/wal/instance
I20260808 22:31:58.603943 32094 external_mini_cluster.cc:1428] Setting key 2da1acde5944e315a915e61f8a6db2ae
I20260808 22:31:58.607314 8317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0a5a49c7d18482ba56669b2fad3a41a: Bootstrap starting.
I20260808 22:31:58.609444 8317 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0a5a49c7d18482ba56669b2fad3a41a: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:58.610129 8317 log.cc:824] T 00000000000000000000000000000000 P d0a5a49c7d18482ba56669b2fad3a41a: Log is configured to *not* fsync() on all Append() calls
I20260808 22:31:58.611850 8317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0a5a49c7d18482ba56669b2fad3a41a: No bootstrap required, opened a new log
I20260808 22:31:58.612545 8320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:58.605210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39044 (local address 127.31.87.190:44333)
0808 22:31:58.605803 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:58.605817 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:58.605851 (+ 34us) server_negotiation.cc:407] Connection header received
0808 22:31:58.606539 (+ 688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:58.606557 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:58.606958 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:58.607266 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:58.607672 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:58.608613 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:58.609551 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:58.610018 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:58.610172 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:58.610927 (+ 755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:58.610932 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:58.610945 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:31:58.611309 (+ 364us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:58.611362 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:31:58.611392 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:58.611476 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:58.611616 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 22:31:58.611858 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":113,"threads_started":1}
I20260808 22:31:58.615233 32094 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:31:58.615403 8317 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0a5a49c7d18482ba56669b2fad3a41a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0a5a49c7d18482ba56669b2fad3a41a" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 44333 } }
I20260808 22:31:58.615608 8317 raft_consensus.cc:374] T 00000000000000000000000000000000 P d0a5a49c7d18482ba56669b2fad3a41a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:58.615661 8317 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0a5a49c7d18482ba56669b2fad3a41a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0a5a49c7d18482ba56669b2fad3a41a, State: Initialized, Role: FOLLOWER
I20260808 22:31:58.616088 8317 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0a5a49c7d18482ba56669b2fad3a41a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0a5a49c7d18482ba56669b2fad3a41a" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 44333 } }
I20260808 22:31:58.616255 8317 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0a5a49c7d18482ba56669b2fad3a41a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:31:58.616341 8317 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0a5a49c7d18482ba56669b2fad3a41a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:31:58.616442 8317 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0a5a49c7d18482ba56669b2fad3a41a [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:58.617401 8317 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0a5a49c7d18482ba56669b2fad3a41a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0a5a49c7d18482ba56669b2fad3a41a" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 44333 } }
I20260808 22:31:58.617679 8317 leader_election.cc:302] T 00000000000000000000000000000000 P d0a5a49c7d18482ba56669b2fad3a41a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0a5a49c7d18482ba56669b2fad3a41a; no voters:
I20260808 22:31:58.617913 8317 leader_election.cc:288] T 00000000000000000000000000000000 P d0a5a49c7d18482ba56669b2fad3a41a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:31:58.618006 8325 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0a5a49c7d18482ba56669b2fad3a41a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:31:58.618216 8325 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0a5a49c7d18482ba56669b2fad3a41a [term 1 LEADER]: Becoming Leader. State: Replica: d0a5a49c7d18482ba56669b2fad3a41a, State: Running, Role: LEADER
I20260808 22:31:58.618513 8325 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0a5a49c7d18482ba56669b2fad3a41a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0a5a49c7d18482ba56669b2fad3a41a" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 44333 } }
I20260808 22:31:58.618988 8317 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0a5a49c7d18482ba56669b2fad3a41a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:31:58.620213 8327 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0a5a49c7d18482ba56669b2fad3a41a [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0a5a49c7d18482ba56669b2fad3a41a. Latest consensus state: current_term: 1 leader_uuid: "d0a5a49c7d18482ba56669b2fad3a41a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0a5a49c7d18482ba56669b2fad3a41a" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 44333 } } }
I20260808 22:31:58.620247 8326 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0a5a49c7d18482ba56669b2fad3a41a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d0a5a49c7d18482ba56669b2fad3a41a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0a5a49c7d18482ba56669b2fad3a41a" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 44333 } } }
I20260808 22:31:58.620316 8327 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0a5a49c7d18482ba56669b2fad3a41a [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:58.620350 8326 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0a5a49c7d18482ba56669b2fad3a41a [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:58.620723 8342 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:31:58.623360 8342 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:31:58.628072 8342 catalog_manager.cc:1347] Generated new cluster ID: 04ed7eaf5b5c477395196ce4fd784513
I20260808 22:31:58.628130 8342 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:31:58.642004 8342 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:31:58.643096 8342 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:31:58.646934 8342 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0a5a49c7d18482ba56669b2fad3a41a: Generated new TSK 0
I20260808 22:31:58.647601 8342 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:31:58.661505 8320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:58.656750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39056 (local address 127.31.87.190:44333)
0808 22:31:58.656871 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:58.656876 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:58.657017 (+ 141us) server_negotiation.cc:407] Connection header received
0808 22:31:58.657172 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:58.657175 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:58.657228 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:58.657347 (+ 119us) server_negotiation.cc:226] Negotiated authn=JWT
0808 22:31:58.657536 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:58.658353 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:58.659141 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:58.659479 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:58.659574 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:58.659790 (+ 216us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 22:31:58.661143 (+ 1353us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44069/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 22:31:58.661367 (+ 224us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39056: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44069/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":47}
W20260808 22:31:58.661569 8320 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39056: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44069/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260808 22:31:58.663285 8358 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:31:58.656643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:44333 (local address 127.0.0.1:39056)
0808 22:31:58.656944 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:58.656956 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:58.657045 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:58.657342 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:58.657344 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:58.657351 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0808 22:31:58.657444 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:58.657455 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:58.658441 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:58.658443 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:58.659044 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:58.659050 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:58.659574 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:58.659576 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:58.659635 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:58.659656 (+ 21us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 22:31:58.661297 (+ 1641us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44069/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 22:31:58.663116 (+ 1819us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.31.87.190:44333: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44069/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":229,"mutex_wait_us":1641,"thread_start_us":113,"threads_started":1}
I20260808 22:31:58.663930 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 8221
E20260808 22:31:58.678471 8359 webserver.cc:533] Webserver: sslize failed:
E20260808 22:31:58.678544 8359 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-08T22:31:58Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (290 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-08T22:31:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:31:58Z Disabled control of system clock
I20260808 22:31:58.718623 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:37235
--webserver_interface=127.31.87.190
--webserver_port=0
--builtin_ntp_servers=127.31.87.148:36233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:37235
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786228291013016-32094-0/testchainca.cert with env {}
W20260808 22:31:58.827880 8370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:58.828127 8370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:58.828167 8370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:58.831996 8370 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:31:58.832046 8370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:58.832068 8370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:58.832098 8370 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:31:58.832114 8370 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 22:31:58.832139 8370 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 22:31:58.832160 8370 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260808 22:31:58.837302 8370 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:36233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786228291013016-32094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:37235
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:37235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.87.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786228291013016-32094-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8370
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:58.838255 8370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:58.839174 8370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 22:31:58.847270 8370 server_base.cc:1034] running on GCE node
W20260808 22:31:58.847241 8377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:58.847232 8382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:58.847230 8378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:58.847970 8370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:58.848994 8370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:58.850163 8370 hybrid_clock.cc:648] HybridClock initialized: now 1786228318850137 us; error 34 us; skew 500 ppm
I20260808 22:31:58.851913 8370 webserver.cc:460] Webserver started at http://127.31.87.190:40707/ using document root <none> and password file <none>
I20260808 22:31:58.852479 8370 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:58.852548 8370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:58.852771 8370 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:58.854866 8370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786228291013016-32094-0/minicluster-data/master-0/data/instance:
uuid: "d111ab57e96242a7a2cb8c057f8dcf64"
format_stamp: "Formatted at 2026-08-08 22:31:58 on dist-test-slave-266d"
server_key: "4e2a55fc64e924f97b9a2a83bbbd456b"
server_key_iv: "b7ae38e4ba71558c0eb422111dd012d7"
server_key_version: "encryptionkey@0"
I20260808 22:31:58.855470 8370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786228291013016-32094-0/minicluster-data/master-0/wal/instance:
uuid: "d111ab57e96242a7a2cb8c057f8dcf64"
format_stamp: "Formatted at 2026-08-08 22:31:58 on dist-test-slave-266d"
server_key: "4e2a55fc64e924f97b9a2a83bbbd456b"
server_key_iv: "b7ae38e4ba71558c0eb422111dd012d7"
server_key_version: "encryptionkey@0"
I20260808 22:31:58.859197 8370 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260808 22:31:58.861498 8392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:58.862632 8370 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:31:58.862747 8370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "d111ab57e96242a7a2cb8c057f8dcf64"
format_stamp: "Formatted at 2026-08-08 22:31:58 on dist-test-slave-266d"
server_key: "4e2a55fc64e924f97b9a2a83bbbd456b"
server_key_iv: "b7ae38e4ba71558c0eb422111dd012d7"
server_key_version: "encryptionkey@0"
I20260808 22:31:58.862843 8370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:58.877131 8370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:58.877766 8370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:58.877946 8370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:58.885658 8370 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.190:37235
I20260808 22:31:58.885659 8478 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.190:37235 every 8 connection(s)
I20260808 22:31:58.886615 8370 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:31:58.889425 8479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:58.893918 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 8370
I20260808 22:31:58.894022 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786228291013016-32094-0/minicluster-data/master-0/wal/instance
I20260808 22:31:58.894271 32094 external_mini_cluster.cc:1428] Setting key 64007fd64ec30ed351b000a991976f41
I20260808 22:31:58.895028 8479 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d111ab57e96242a7a2cb8c057f8dcf64: Bootstrap starting.
I20260808 22:31:58.897775 8479 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d111ab57e96242a7a2cb8c057f8dcf64: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:58.898660 8479 log.cc:824] T 00000000000000000000000000000000 P d111ab57e96242a7a2cb8c057f8dcf64: Log is configured to *not* fsync() on all Append() calls
I20260808 22:31:58.900650 8479 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d111ab57e96242a7a2cb8c057f8dcf64: No bootstrap required, opened a new log
I20260808 22:31:58.904299 8485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:58.895568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34314 (local address 127.31.87.190:37235)
0808 22:31:58.896275 (+ 707us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:58.896294 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:58.896350 (+ 56us) server_negotiation.cc:407] Connection header received
0808 22:31:58.897179 (+ 829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:58.897203 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:58.897696 (+ 493us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:58.898086 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:58.898525 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:58.899846 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:58.900708 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:58.901416 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:58.901574 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:58.902284 (+ 710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:58.902290 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:58.902307 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:31:58.902786 (+ 479us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:58.902857 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:31:58.902885 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:58.902982 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:58.903050 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 22:31:58.903326 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":110,"threads_started":1}
I20260808 22:31:58.904896 8479 raft_consensus.cc:348] T 00000000000000000000000000000000 P d111ab57e96242a7a2cb8c057f8dcf64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d111ab57e96242a7a2cb8c057f8dcf64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 37235 } }
I20260808 22:31:58.905141 8479 raft_consensus.cc:374] T 00000000000000000000000000000000 P d111ab57e96242a7a2cb8c057f8dcf64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:58.905201 8479 raft_consensus.cc:729] T 00000000000000000000000000000000 P d111ab57e96242a7a2cb8c057f8dcf64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d111ab57e96242a7a2cb8c057f8dcf64, State: Initialized, Role: FOLLOWER
I20260808 22:31:58.905676 8479 consensus_queue.cc:260] T 00000000000000000000000000000000 P d111ab57e96242a7a2cb8c057f8dcf64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d111ab57e96242a7a2cb8c057f8dcf64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 37235 } }
I20260808 22:31:58.905833 8479 raft_consensus.cc:388] T 00000000000000000000000000000000 P d111ab57e96242a7a2cb8c057f8dcf64 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:31:58.905895 8479 raft_consensus.cc:482] T 00000000000000000000000000000000 P d111ab57e96242a7a2cb8c057f8dcf64 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:31:58.905988 8479 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d111ab57e96242a7a2cb8c057f8dcf64 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:58.906991 8479 raft_consensus.cc:504] T 00000000000000000000000000000000 P d111ab57e96242a7a2cb8c057f8dcf64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d111ab57e96242a7a2cb8c057f8dcf64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 37235 } }
I20260808 22:31:58.907207 32094 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:31:58.907339 8479 leader_election.cc:302] T 00000000000000000000000000000000 P d111ab57e96242a7a2cb8c057f8dcf64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d111ab57e96242a7a2cb8c057f8dcf64; no voters:
I20260808 22:31:58.907652 8479 leader_election.cc:288] T 00000000000000000000000000000000 P d111ab57e96242a7a2cb8c057f8dcf64 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:31:58.907739 8488 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d111ab57e96242a7a2cb8c057f8dcf64 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:31:58.907905 8488 raft_consensus.cc:686] T 00000000000000000000000000000000 P d111ab57e96242a7a2cb8c057f8dcf64 [term 1 LEADER]: Becoming Leader. State: Replica: d111ab57e96242a7a2cb8c057f8dcf64, State: Running, Role: LEADER
I20260808 22:31:58.908203 8488 consensus_queue.cc:237] T 00000000000000000000000000000000 P d111ab57e96242a7a2cb8c057f8dcf64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d111ab57e96242a7a2cb8c057f8dcf64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 37235 } }
I20260808 22:31:58.908905 8479 sys_catalog.cc:564] T 00000000000000000000000000000000 P d111ab57e96242a7a2cb8c057f8dcf64 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:31:58.909855 8489 sys_catalog.cc:455] T 00000000000000000000000000000000 P d111ab57e96242a7a2cb8c057f8dcf64 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d111ab57e96242a7a2cb8c057f8dcf64. Latest consensus state: current_term: 1 leader_uuid: "d111ab57e96242a7a2cb8c057f8dcf64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d111ab57e96242a7a2cb8c057f8dcf64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 37235 } } }
I20260808 22:31:58.910028 8489 sys_catalog.cc:458] T 00000000000000000000000000000000 P d111ab57e96242a7a2cb8c057f8dcf64 [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:58.910454 8501 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:31:58.910768 8491 sys_catalog.cc:455] T 00000000000000000000000000000000 P d111ab57e96242a7a2cb8c057f8dcf64 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d111ab57e96242a7a2cb8c057f8dcf64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d111ab57e96242a7a2cb8c057f8dcf64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 37235 } } }
I20260808 22:31:58.910878 8491 sys_catalog.cc:458] T 00000000000000000000000000000000 P d111ab57e96242a7a2cb8c057f8dcf64 [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:58.913549 8501 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:31:58.918401 8501 catalog_manager.cc:1347] Generated new cluster ID: f6e890be1a7c4b06b0d518c5ea2f432a
I20260808 22:31:58.918462 8501 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:31:58.951329 8501 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:31:58.952517 8501 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:31:58.960268 8501 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d111ab57e96242a7a2cb8c057f8dcf64: Generated new TSK 0
I20260808 22:31:58.960903 8501 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:31:59.012131 8485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:58.979465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34320 (local address 127.31.87.190:37235)
0808 22:31:58.979593 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:58.979597 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:58.979800 (+ 203us) server_negotiation.cc:407] Connection header received
0808 22:31:58.979969 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:58.979972 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:58.980026 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:58.980144 (+ 118us) server_negotiation.cc:226] Negotiated authn=JWT
0808 22:31:58.980405 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:58.981038 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:58.982303 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:58.982516 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:58.982592 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:58.982912 (+ 320us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 22:31:59.011672 (+ 28760us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 22:31:59.011981 (+ 309us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34320: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":49}
W20260808 22:31:59.012215 8485 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34320: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260808 22:31:59.012389 8525 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:31:58.979379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37235 (local address 127.0.0.1:34320)
0808 22:31:58.979701 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:58.979718 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:58.979843 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:58.980149 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:58.980153 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:58.980162 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0808 22:31:58.980287 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:58.980294 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:58.981209 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:58.981214 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:58.982185 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:58.982196 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:58.982634 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:58.982639 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:58.982737 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:58.982768 (+ 31us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 22:31:59.011936 (+ 29168us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 22:31:59.012182 (+ 246us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.31.87.190:37235: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":100,"threads_started":1}
I20260808 22:31:59.013134 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 8370
2026-08-08T22:31:59Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (335 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (120 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (119 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:31:59 dist-test-slave-266d krb5kdc[8542](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:31:59 dist-test-slave-266d krb5kdc[8542](info): set up 2 sockets
Aug 08 22:31:59 dist-test-slave-266d krb5kdc[8542](info): commencing operation
krb5kdc: starting...
W20260808 22:32:00.390978 32094 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Aug 08 22:32:00 dist-test-slave-266d krb5kdc[8542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228320, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:32:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:32:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.87.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
I20260808 22:32:00.507258 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:46695
--webserver_interface=127.31.87.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:42031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:46695
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:32:00.615062 8565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:32:00.615296 8565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:32:00.615337 8565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:32:00.619256 8565 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:32:00.619313 8565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:32:00.619333 8565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:32:00.619359 8565 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:32:00.619382 8565 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:32:00.624361 8565 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:42031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:46695
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:46695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.87.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8565
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:32:00.625365 8565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:32:00.626340 8565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:32:00.634904 8575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:00.634898 8574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:00.634898 8577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:00.635110 8565 server_base.cc:1034] running on GCE node
I20260808 22:32:00.635794 8565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:32:00.636921 8565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:32:00.638083 8565 hybrid_clock.cc:648] HybridClock initialized: now 1786228320638063 us; error 55 us; skew 500 ppm
Aug 08 22:32:00 dist-test-slave-266d krb5kdc[8542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228320, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:32:00.640827 8565 init.cc:377] Logged in from keytab as kudu/127.31.87.190@KRBTEST.COM (short username kudu)
I20260808 22:32:00.641729 8565 webserver.cc:460] Webserver started at http://127.31.87.190:33099/ using document root <none> and password file <none>
I20260808 22:32:00.642256 8565 fs_manager.cc:359] Metadata directory not provided
I20260808 22:32:00.642313 8565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:32:00.642481 8565 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:32:00.644447 8565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/master-0/data/instance:
uuid: "97a2028a11b34edeb17f54dba8d30ed6"
format_stamp: "Formatted at 2026-08-08 22:32:00 on dist-test-slave-266d"
server_key: "160fdb136c815b21e927543573db28c9"
server_key_iv: "ac03256e852f14ea06f8394f9b7e373c"
server_key_version: "encryptionkey@0"
I20260808 22:32:00.644958 8565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/master-0/wal/instance:
uuid: "97a2028a11b34edeb17f54dba8d30ed6"
format_stamp: "Formatted at 2026-08-08 22:32:00 on dist-test-slave-266d"
server_key: "160fdb136c815b21e927543573db28c9"
server_key_iv: "ac03256e852f14ea06f8394f9b7e373c"
server_key_version: "encryptionkey@0"
I20260808 22:32:00.648404 8565 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 22:32:00.650672 8587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:00.651787 8565 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:32:00.651892 8565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "97a2028a11b34edeb17f54dba8d30ed6"
format_stamp: "Formatted at 2026-08-08 22:32:00 on dist-test-slave-266d"
server_key: "160fdb136c815b21e927543573db28c9"
server_key_iv: "ac03256e852f14ea06f8394f9b7e373c"
server_key_version: "encryptionkey@0"
I20260808 22:32:00.651983 8565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:32:00.683933 8565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:32:00.687018 8565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:32:00.687193 8565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:32:00.693723 8565 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.190:46695
I20260808 22:32:00.693744 8662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.190:46695 every 8 connection(s)
I20260808 22:32:00.694584 8565 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:32:00.697198 8663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:32:00.702343 8663 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97a2028a11b34edeb17f54dba8d30ed6: Bootstrap starting.
I20260808 22:32:00.702423 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 8565
I20260808 22:32:00.702543 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/master-0/wal/instance
I20260808 22:32:00.702805 32094 external_mini_cluster.cc:1428] Setting key 3c25f13946ab710bc30d7e1f59f102e3
I20260808 22:32:00.704643 8663 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97a2028a11b34edeb17f54dba8d30ed6: Neither blocks nor log segments found. Creating new log.
I20260808 22:32:00.705376 8663 log.cc:824] T 00000000000000000000000000000000 P 97a2028a11b34edeb17f54dba8d30ed6: Log is configured to *not* fsync() on all Append() calls
I20260808 22:32:00.707247 8663 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97a2028a11b34edeb17f54dba8d30ed6: No bootstrap required, opened a new log
I20260808 22:32:00.710947 8663 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97a2028a11b34edeb17f54dba8d30ed6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97a2028a11b34edeb17f54dba8d30ed6" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 46695 } }
I20260808 22:32:00.711145 8663 raft_consensus.cc:374] T 00000000000000000000000000000000 P 97a2028a11b34edeb17f54dba8d30ed6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:32:00.711200 8663 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97a2028a11b34edeb17f54dba8d30ed6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97a2028a11b34edeb17f54dba8d30ed6, State: Initialized, Role: FOLLOWER
I20260808 22:32:00.711575 8663 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97a2028a11b34edeb17f54dba8d30ed6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97a2028a11b34edeb17f54dba8d30ed6" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 46695 } }
I20260808 22:32:00.711695 8663 raft_consensus.cc:388] T 00000000000000000000000000000000 P 97a2028a11b34edeb17f54dba8d30ed6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:32:00.711757 8663 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97a2028a11b34edeb17f54dba8d30ed6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:32:00.711860 8663 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97a2028a11b34edeb17f54dba8d30ed6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:32:00.712612 8663 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97a2028a11b34edeb17f54dba8d30ed6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97a2028a11b34edeb17f54dba8d30ed6" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 46695 } }
I20260808 22:32:00.712917 8663 leader_election.cc:302] T 00000000000000000000000000000000 P 97a2028a11b34edeb17f54dba8d30ed6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97a2028a11b34edeb17f54dba8d30ed6; no voters:
I20260808 22:32:00.713195 8663 leader_election.cc:288] T 00000000000000000000000000000000 P 97a2028a11b34edeb17f54dba8d30ed6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:32:00.713269 8671 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97a2028a11b34edeb17f54dba8d30ed6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:32:00.713515 8671 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97a2028a11b34edeb17f54dba8d30ed6 [term 1 LEADER]: Becoming Leader. State: Replica: 97a2028a11b34edeb17f54dba8d30ed6, State: Running, Role: LEADER
I20260808 22:32:00.713832 8671 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97a2028a11b34edeb17f54dba8d30ed6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97a2028a11b34edeb17f54dba8d30ed6" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 46695 } }
I20260808 22:32:00.714387 8663 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97a2028a11b34edeb17f54dba8d30ed6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:32:00.715430 8673 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97a2028a11b34edeb17f54dba8d30ed6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "97a2028a11b34edeb17f54dba8d30ed6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97a2028a11b34edeb17f54dba8d30ed6" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 46695 } } }
I20260808 22:32:00.715574 8673 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97a2028a11b34edeb17f54dba8d30ed6 [sys.catalog]: This master's current role is: LEADER
I20260808 22:32:00.715435 8675 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97a2028a11b34edeb17f54dba8d30ed6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97a2028a11b34edeb17f54dba8d30ed6. Latest consensus state: current_term: 1 leader_uuid: "97a2028a11b34edeb17f54dba8d30ed6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97a2028a11b34edeb17f54dba8d30ed6" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 46695 } } }
Aug 08 22:32:00 dist-test-slave-266d krb5kdc[8542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228320, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:32:00.715835 8675 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97a2028a11b34edeb17f54dba8d30ed6 [sys.catalog]: This master's current role is: LEADER
I20260808 22:32:00.715909 8683 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:32:00.718708 8683 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:32:00.721810 8666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:00.704050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33112 (local address 127.31.87.190:46695)
0808 22:32:00.704440 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:00.704454 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:00.704488 (+ 34us) server_negotiation.cc:407] Connection header received
0808 22:32:00.705115 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:00.705138 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:00.705439 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:00.705738 (+ 299us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:00.706526 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:00.707485 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:00.708601 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:00.709028 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:00.709173 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:00.716506 (+ 7333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:00.716514 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:00.716531 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:00.716563 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:00.718791 (+ 2228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:00.719275 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:00.719280 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:00.719286 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:00.719419 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:00.719688 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:00.719692 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:00.719693 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:00.720175 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:00.720358 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:00.720620 (+ 262us) server_negotiation.cc:299] Negotiation successful
0808 22:32:00.720896 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":100,"threads_started":1}
I20260808 22:32:00.724295 8683 catalog_manager.cc:1347] Generated new cluster ID: 596466f943bb4401af491d816c95a99a
I20260808 22:32:00.724354 8683 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:32:00.741357 8683 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:32:00.742452 8683 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:32:00.749709 8683 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 97a2028a11b34edeb17f54dba8d30ed6: Generated new TSK 0
I20260808 22:32:00.750372 8683 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.87.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab.
Entry for principal kudu/127.31.87.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab.
I20260808 22:32:00.779842 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.129:0
--local_ip_for_outbound_sockets=127.31.87.129
--webserver_interface=127.31.87.129
--webserver_port=0
--tserver_master_addrs=127.31.87.190:46695
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab
--principal=kudu/127.31.87.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:42031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:32:00.888182 8695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:32:00.888499 8695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:32:00.888559 8695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:32:00.893018 8695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:32:00.893071 8695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:32:00.893167 8695 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.129
I20260808 22:32:00.898252 8695 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:42031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab
--principal=kudu/127.31.87.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.87.129
--webserver_port=0
--tserver_master_addrs=127.31.87.190:46695
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.129
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:32:00.899178 8695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:32:00.900101 8695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:32:00.909161 8702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:00.909219 8703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:00.909168 8706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:00.909446 8695 server_base.cc:1034] running on GCE node
I20260808 22:32:00.909786 8695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:32:00.910346 8695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:32:00.911487 8695 hybrid_clock.cc:648] HybridClock initialized: now 1786228320911464 us; error 35 us; skew 500 ppm
Aug 08 22:32:00 dist-test-slave-266d krb5kdc[8542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228320, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:32:00.914068 8695 init.cc:377] Logged in from keytab as kudu/127.31.87.129@KRBTEST.COM (short username kudu)
I20260808 22:32:00.914849 8695 webserver.cc:460] Webserver started at http://127.31.87.129:42579/ using document root <none> and password file <none>
I20260808 22:32:00.915339 8695 fs_manager.cc:359] Metadata directory not provided
I20260808 22:32:00.915395 8695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:32:00.915550 8695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:32:00.917500 8695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/ts-0/data/instance:
uuid: "6717ea62b1d6474b8ee36de5792ac63a"
format_stamp: "Formatted at 2026-08-08 22:32:00 on dist-test-slave-266d"
server_key: "b7b18de04ec6dc3a67741092cc06fb40"
server_key_iv: "a07349f68bd818441237c2fc09cbca28"
server_key_version: "encryptionkey@0"
I20260808 22:32:00.917987 8695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/ts-0/wal/instance:
uuid: "6717ea62b1d6474b8ee36de5792ac63a"
format_stamp: "Formatted at 2026-08-08 22:32:00 on dist-test-slave-266d"
server_key: "b7b18de04ec6dc3a67741092cc06fb40"
server_key_iv: "a07349f68bd818441237c2fc09cbca28"
server_key_version: "encryptionkey@0"
I20260808 22:32:00.921368 8695 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:32:00.923588 8722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:00.924676 8695 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:32:00.924783 8695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/ts-0/wal
uuid: "6717ea62b1d6474b8ee36de5792ac63a"
format_stamp: "Formatted at 2026-08-08 22:32:00 on dist-test-slave-266d"
server_key: "b7b18de04ec6dc3a67741092cc06fb40"
server_key_iv: "a07349f68bd818441237c2fc09cbca28"
server_key_version: "encryptionkey@0"
I20260808 22:32:00.924875 8695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:32:00.936686 8695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:32:00.939476 8695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:32:00.939635 8695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:32:00.940156 8695 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:32:00.940896 8695 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:32:00.940941 8695 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:00.940980 8695 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:32:00.941008 8695 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:00.948859 8695 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.129:39647
I20260808 22:32:00.948880 8893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.129:39647 every 8 connection(s)
I20260808 22:32:00.949710 8695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
I20260808 22:32:00.955307 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 8695
I20260808 22:32:00.955415 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/ts-0/wal/instance
I20260808 22:32:00.955725 32094 external_mini_cluster.cc:1428] Setting key 9d9ba7ca64ecf6104d5e3ab8e62cd16a
Aug 08 22:32:00 dist-test-slave-266d krb5kdc[8542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228320, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.129@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:32:00.963124 8666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:00.951334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:43809 (local address 127.31.87.190:46695)
0808 22:32:00.951470 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:00.951475 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:00.952256 (+ 781us) server_negotiation.cc:407] Connection header received
0808 22:32:00.953785 (+ 1529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:00.953789 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:00.953836 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:00.953897 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:00.954994 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:00.955590 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:00.956914 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:00.957261 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:00.957345 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:00.959799 (+ 2454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:00.959802 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:00.959804 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:00.959826 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:00.961234 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:00.961680 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:00.961683 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:00.961685 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:00.961772 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:00.962053 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:00.962057 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:00.962058 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:00.962286 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:00.962385 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:00.962814 (+ 429us) server_negotiation.cc:299] Negotiation successful
0808 22:32:00.962937 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 22:32:00.963759 8897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:00.951687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:46695 (local address 127.31.87.129:43809)
0808 22:32:00.952095 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:00.952139 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:00.953588 (+ 1449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:00.954030 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:00.954036 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:00.954346 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:00.954839 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:00.954853 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:00.955708 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:00.955712 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:00.956795 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:00.956805 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:00.957403 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:00.957407 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:00.957601 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:00.958129 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:00.958148 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:00.959645 (+ 1497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:32:00.961336 (+ 1691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:00.961341 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:00.961352 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:00.961585 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:00.961852 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:00.961854 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:00.961856 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:00.961964 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:00.962380 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:32:00.962385 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:32:00.962596 (+ 211us) client_negotiation.cc:769] Sending connection context
0808 22:32:00.962740 (+ 144us) client_negotiation.cc:240] Negotiation successful
0808 22:32:00.962976 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":106,"threads_started":1}
I20260808 22:32:00.964877 8894 heartbeater.cc:347] Connected to a master server at 127.31.87.190:46695
I20260808 22:32:00.965112 8894 heartbeater.cc:464] Registering TS with master...
I20260808 22:32:00.965580 8894 heartbeater.cc:511] Master 127.31.87.190:46695 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.31.87.130@KRBTEST.COM; defaulting to no policy
I20260808 22:32:00.966998 8611 ts_manager.cc:194] Registered new tserver with Master: 6717ea62b1d6474b8ee36de5792ac63a (127.31.87.129:39647)
I20260808 22:32:00.968078 8611 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.129@KRBTEST.COM'} at 127.31.87.129:43809
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab.
Entry for principal kudu/127.31.87.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab.
I20260808 22:32:00.981666 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.130:0
--local_ip_for_outbound_sockets=127.31.87.130
--webserver_interface=127.31.87.130
--webserver_port=0
--tserver_master_addrs=127.31.87.190:46695
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab
--principal=kudu/127.31.87.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:42031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:32:01.088469 8901 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:32:01.088697 8901 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:32:01.088737 8901 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:32:01.092545 8901 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:32:01.092588 8901 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:32:01.092646 8901 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.130
I20260808 22:32:01.097539 8901 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:42031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab
--principal=kudu/127.31.87.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.87.130
--webserver_port=0
--tserver_master_addrs=127.31.87.190:46695
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8901
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.130
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:32:01.098389 8901 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:32:01.099257 8901 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:32:01.108176 8909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:01.108242 8901 server_base.cc:1034] running on GCE node
W20260808 22:32:01.108196 8908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:01.108191 8911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:01.108814 8901 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:32:01.109421 8901 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:32:01.110567 8901 hybrid_clock.cc:648] HybridClock initialized: now 1786228321110544 us; error 35 us; skew 500 ppm
Aug 08 22:32:01 dist-test-slave-266d krb5kdc[8542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228321, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:32:01.113327 8901 init.cc:377] Logged in from keytab as kudu/127.31.87.130@KRBTEST.COM (short username kudu)
I20260808 22:32:01.114110 8901 webserver.cc:460] Webserver started at http://127.31.87.130:39129/ using document root <none> and password file <none>
I20260808 22:32:01.114630 8901 fs_manager.cc:359] Metadata directory not provided
I20260808 22:32:01.114687 8901 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:32:01.114845 8901 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:32:01.116928 8901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/ts-1/data/instance:
uuid: "1a2c5f4a8e994656963e1bfd59f32bc3"
format_stamp: "Formatted at 2026-08-08 22:32:01 on dist-test-slave-266d"
server_key: "25bd936433560cbfc81850f71cc6b9d4"
server_key_iv: "e21977c1f906d859b4e33cacc5124b07"
server_key_version: "encryptionkey@0"
I20260808 22:32:01.117504 8901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/ts-1/wal/instance:
uuid: "1a2c5f4a8e994656963e1bfd59f32bc3"
format_stamp: "Formatted at 2026-08-08 22:32:01 on dist-test-slave-266d"
server_key: "25bd936433560cbfc81850f71cc6b9d4"
server_key_iv: "e21977c1f906d859b4e33cacc5124b07"
server_key_version: "encryptionkey@0"
I20260808 22:32:01.121073 8901 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 22:32:01.123255 8925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:01.124341 8901 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 22:32:01.124444 8901 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/ts-1/wal
uuid: "1a2c5f4a8e994656963e1bfd59f32bc3"
format_stamp: "Formatted at 2026-08-08 22:32:01 on dist-test-slave-266d"
server_key: "25bd936433560cbfc81850f71cc6b9d4"
server_key_iv: "e21977c1f906d859b4e33cacc5124b07"
server_key_version: "encryptionkey@0"
I20260808 22:32:01.124534 8901 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:32:01.136819 8901 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:32:01.139262 8901 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:32:01.139417 8901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:32:01.139922 8901 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:32:01.140676 8901 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:32:01.140722 8901 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:01.140762 8901 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:32:01.140791 8901 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:01.148775 8901 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.130:42317
I20260808 22:32:01.148804 9112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.130:42317 every 8 connection(s)
I20260808 22:32:01.149577 8901 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
I20260808 22:32:01.157051 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 8901
I20260808 22:32:01.157191 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/ts-1/wal/instance
I20260808 22:32:01.157444 32094 external_mini_cluster.cc:1428] Setting key 0f97b94e197c2695e2327add36ec93fe
Aug 08 22:32:01 dist-test-slave-266d krb5kdc[8542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228321, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.130@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:32:01.161979 8666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:01.151144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:48047 (local address 127.31.87.190:46695)
0808 22:32:01.151273 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:01.151278 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:01.152050 (+ 772us) server_negotiation.cc:407] Connection header received
0808 22:32:01.153325 (+ 1275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:01.153328 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:01.153373 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:01.153441 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:01.154440 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:01.154935 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:01.155973 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:01.156198 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:01.156320 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:01.158640 (+ 2320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:01.158643 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:01.158645 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:01.158670 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:01.160052 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:01.160503 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:01.160507 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:01.160508 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:01.160594 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:01.160961 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:01.160964 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:01.160965 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:01.161182 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:01.161259 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:01.161765 (+ 506us) server_negotiation.cc:299] Negotiation successful
0808 22:32:01.161860 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 22:32:01.162928 9119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:01.151504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:46695 (local address 127.31.87.130:48047)
0808 22:32:01.151917 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:01.151955 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:01.153151 (+ 1196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:01.153584 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:01.153590 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:01.153911 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:01.154330 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:01.154343 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:01.155018 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:01.155020 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:01.155888 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:01.155897 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:01.156356 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:01.156359 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:01.156492 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:01.157003 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:01.157018 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:01.158484 (+ 1466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:32:01.160170 (+ 1686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:01.160175 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:01.160185 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:01.160418 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:01.160681 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:01.160684 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:01.160686 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:01.160868 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:01.161272 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:32:01.161277 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:32:01.161549 (+ 272us) client_negotiation.cc:769] Sending connection context
0808 22:32:01.161742 (+ 193us) client_negotiation.cc:240] Negotiation successful
0808 22:32:01.162055 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":105,"threads_started":1}
I20260808 22:32:01.164018 9115 heartbeater.cc:347] Connected to a master server at 127.31.87.190:46695
I20260808 22:32:01.164251 9115 heartbeater.cc:464] Registering TS with master...
I20260808 22:32:01.164695 9115 heartbeater.cc:511] Master 127.31.87.190:46695 requested a full tablet report, sending...
I20260808 22:32:01.165689 8611 ts_manager.cc:194] Registered new tserver with Master: 1a2c5f4a8e994656963e1bfd59f32bc3 (127.31.87.130:42317)
I20260808 22:32:01.166217 8611 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.130@KRBTEST.COM'} at 127.31.87.130:48047
WARNING: no policy specified for kudu/127.31.87.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab.
Entry for principal kudu/127.31.87.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab.
I20260808 22:32:01.183283 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.131:0
--local_ip_for_outbound_sockets=127.31.87.131
--webserver_interface=127.31.87.131
--webserver_port=0
--tserver_master_addrs=127.31.87.190:46695
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab
--principal=kudu/127.31.87.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:42031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:32:01.290066 9122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:32:01.290292 9122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:32:01.290334 9122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:32:01.294210 9122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:32:01.294258 9122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:32:01.294322 9122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.131
I20260808 22:32:01.299311 9122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:42031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab
--principal=kudu/127.31.87.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.87.131
--webserver_port=0
--tserver_master_addrs=127.31.87.190:46695
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.131
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:32:01.300202 9122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:32:01.301152 9122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:32:01.309947 9129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:01.309935 9131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:01.310040 9128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:01.310469 9122 server_base.cc:1034] running on GCE node
I20260808 22:32:01.310815 9122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:32:01.311385 9122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:32:01.312541 9122 hybrid_clock.cc:648] HybridClock initialized: now 1786228321312515 us; error 37 us; skew 500 ppm
Aug 08 22:32:01 dist-test-slave-266d krb5kdc[8542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228321, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:32:01.316167 9122 init.cc:377] Logged in from keytab as kudu/127.31.87.131@KRBTEST.COM (short username kudu)
I20260808 22:32:01.317219 9122 webserver.cc:460] Webserver started at http://127.31.87.131:46629/ using document root <none> and password file <none>
I20260808 22:32:01.317853 9122 fs_manager.cc:359] Metadata directory not provided
I20260808 22:32:01.317930 9122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:32:01.318184 9122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:32:01.320252 9122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/ts-2/data/instance:
uuid: "582b22a304c44f4088d0910b7d659034"
format_stamp: "Formatted at 2026-08-08 22:32:01 on dist-test-slave-266d"
server_key: "86d9d1e1b7f149baae594183c8e1d8c1"
server_key_iv: "8ef6ea977b5b5b6fc6dca2fbabc2fbcb"
server_key_version: "encryptionkey@0"
I20260808 22:32:01.320868 9122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/ts-2/wal/instance:
uuid: "582b22a304c44f4088d0910b7d659034"
format_stamp: "Formatted at 2026-08-08 22:32:01 on dist-test-slave-266d"
server_key: "86d9d1e1b7f149baae594183c8e1d8c1"
server_key_iv: "8ef6ea977b5b5b6fc6dca2fbabc2fbcb"
server_key_version: "encryptionkey@0"
I20260808 22:32:01.324865 9122 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260808 22:32:01.327142 9142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:01.328392 9122 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 22:32:01.328492 9122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/ts-2/wal
uuid: "582b22a304c44f4088d0910b7d659034"
format_stamp: "Formatted at 2026-08-08 22:32:01 on dist-test-slave-266d"
server_key: "86d9d1e1b7f149baae594183c8e1d8c1"
server_key_iv: "8ef6ea977b5b5b6fc6dca2fbabc2fbcb"
server_key_version: "encryptionkey@0"
I20260808 22:32:01.328581 9122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:32:01.346697 9122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:32:01.349597 9122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:32:01.349757 9122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:32:01.350284 9122 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:32:01.351038 9122 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:32:01.351085 9122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:01.351145 9122 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:32:01.351169 9122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:01.359652 9122 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.131:42493
I20260808 22:32:01.359678 9324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.131:42493 every 8 connection(s)
I20260808 22:32:01.360441 9122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
I20260808 22:32:01.368844 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 9122
I20260808 22:32:01.368938 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/minicluster-data/ts-2/wal/instance
Aug 08 22:32:01 dist-test-slave-266d krb5kdc[8542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228321, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.131@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:32:01.369230 32094 external_mini_cluster.cc:1428] Setting key acf3fbcb9ddb639084736ba9e2cbf2eb
I20260808 22:32:01.372551 8666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:01.362084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:46937 (local address 127.31.87.190:46695)
0808 22:32:01.362203 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:01.362206 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:01.362968 (+ 762us) server_negotiation.cc:407] Connection header received
0808 22:32:01.364274 (+ 1306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:01.364278 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:01.364334 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:01.364403 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:01.365523 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:01.366046 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:01.367114 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:01.367328 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:01.367419 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:01.369566 (+ 2147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:01.369570 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:01.369572 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:01.369595 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:01.370954 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:01.371437 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:01.371439 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:01.371440 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:01.371513 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:01.371755 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:01.371757 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:01.371758 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:01.371910 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:01.371979 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:01.372350 (+ 371us) server_negotiation.cc:299] Negotiation successful
0808 22:32:01.372433 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 22:32:01.373273 9327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:01.362414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:46695 (local address 127.31.87.131:46937)
0808 22:32:01.362835 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:01.362872 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:01.364073 (+ 1201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:01.364569 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:01.364576 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:01.364902 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:01.365383 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:01.365397 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:01.366162 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:01.366165 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:01.366994 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:01.367003 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:01.367459 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:01.367463 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:01.367616 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:01.368052 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:01.368067 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:01.369419 (+ 1352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:32:01.371073 (+ 1654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:01.371078 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:01.371090 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:01.371356 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:01.371579 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:01.371581 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:01.371582 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:01.371693 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:01.371978 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:32:01.371981 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:32:01.372180 (+ 199us) client_negotiation.cc:769] Sending connection context
0808 22:32:01.372315 (+ 135us) client_negotiation.cc:240] Negotiation successful
0808 22:32:01.372523 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":114,"threads_started":1}
I20260808 22:32:01.374239 9325 heartbeater.cc:347] Connected to a master server at 127.31.87.190:46695
I20260808 22:32:01.374425 9325 heartbeater.cc:464] Registering TS with master...
I20260808 22:32:01.374830 9325 heartbeater.cc:511] Master 127.31.87.190:46695 requested a full tablet report, sending...
I20260808 22:32:01.375672 8611 ts_manager.cc:194] Registered new tserver with Master: 582b22a304c44f4088d0910b7d659034 (127.31.87.131:42493)
I20260808 22:32:01.376174 8611 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.131@KRBTEST.COM'} at 127.31.87.131:46937
I20260808 22:32:01.382476 32094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/krb5kdc/test-admin.keytab.
Aug 08 22:32:01 dist-test-slave-266d krb5kdc[8542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:32:01.397320 32094 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260808 22:32:01.399801 9336 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:01.398493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55908 (local address 127.31.87.129:39647)
0808 22:32:01.398800 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:01.398810 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:01.398830 (+ 20us) server_negotiation.cc:407] Connection header received
0808 22:32:01.398874 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:01.398878 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:01.399032 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:01.399142 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:01.399660 (+ 518us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55908: BlockingRecv error: recv got EOF from 127.0.0.1:55908 (error 108)
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":123,"threads_started":1}
W20260808 22:32:01.399976 32094 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/krb5kdc/krb5cc' of type FILE
Aug 08 22:32:01 dist-test-slave-266d krb5kdc[8542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:32:01.400960 32094 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 22:32:01 dist-test-slave-266d krb5kdc[8542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:32:01.401829 32094 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 22:32:01 dist-test-slave-266d krb5kdc[8542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.87.129@KRBTEST.COM
I20260808 22:32:01.410569 9336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:01.402597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55918 (local address 127.31.87.129:39647)
0808 22:32:01.402711 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:01.402715 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:01.402788 (+ 73us) server_negotiation.cc:407] Connection header received
0808 22:32:01.402902 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:01.402906 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:01.402944 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:01.403005 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:01.403512 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:01.404265 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:01.405059 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:01.405454 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:01.405551 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:01.407559 (+ 2008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:01.407568 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:01.407583 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:01.407606 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:01.408879 (+ 1273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:01.409357 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:01.409362 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:01.409363 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:01.409438 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:01.409690 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:01.409694 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:01.409696 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:01.409910 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:01.410112 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:01.410299 (+ 187us) server_negotiation.cc:299] Negotiation successful
0808 22:32:01.410437 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260808 22:32:01.413601 9336 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:01.412846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55932 (local address 127.31.87.129:39647)
0808 22:32:01.412957 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:01.412960 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:01.413025 (+ 65us) server_negotiation.cc:407] Connection header received
0808 22:32:01.413138 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:01.413141 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:01.413174 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:01.413232 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:01.413513 (+ 281us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55932: BlockingRecv error: recv got EOF from 127.0.0.1:55932 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260808 22:32:01.413683 32094 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/krb5kdc/krb5cc' of type FILE
Aug 08 22:32:01 dist-test-slave-266d krb5kdc[8542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:32:01.414507 32094 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 22:32:01 dist-test-slave-266d krb5kdc[8542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:32:01.415365 32094 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 22:32:01 dist-test-slave-266d krb5kdc[8542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.87.129@KRBTEST.COM
I20260808 22:32:01.422729 9336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:01.416075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55934 (local address 127.31.87.129:39647)
0808 22:32:01.416185 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:01.416189 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:01.416253 (+ 64us) server_negotiation.cc:407] Connection header received
0808 22:32:01.416375 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:01.416378 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:01.416416 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:01.416472 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:01.416909 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:01.417469 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:01.418181 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:01.418391 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:01.418473 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:01.420293 (+ 1820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:01.420296 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:01.420298 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:01.420318 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:01.421328 (+ 1010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:01.421714 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:01.421717 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:01.421718 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:01.421805 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:01.422044 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:01.422047 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:01.422049 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:01.422193 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:01.422294 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:01.422509 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 22:32:01.422616 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260808 22:32:01.424947 9336 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:01.424135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55944 (local address 127.31.87.129:39647)
0808 22:32:01.424245 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:01.424248 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:01.424303 (+ 55us) server_negotiation.cc:407] Connection header received
0808 22:32:01.424413 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:01.424416 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:01.424450 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:01.424510 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:01.424862 (+ 352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55944: BlockingRecv error: recv got EOF from 127.0.0.1:55944 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260808 22:32:01.425025 32094 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786228291013016-32094-0/krb5kdc/krb5cc' of type FILE
Aug 08 22:32:01 dist-test-slave-266d krb5kdc[8542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:32:01.425850 32094 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 22:32:01 dist-test-slave-266d krb5kdc[8542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:32:01.426568 32094 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 22:32:01 dist-test-slave-266d krb5kdc[8542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.87.129@KRBTEST.COM
I20260808 22:32:01.433835 9336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:01.427223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55952 (local address 127.31.87.129:39647)
0808 22:32:01.427327 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:01.427331 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:01.427403 (+ 72us) server_negotiation.cc:407] Connection header received
0808 22:32:01.427483 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:01.427485 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:01.427514 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:01.427576 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:01.427996 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:01.428529 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:01.429322 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:01.429515 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:01.429602 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:01.431393 (+ 1791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:01.431396 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:01.431398 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:01.431418 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:01.432547 (+ 1129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:01.432914 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:01.432916 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:01.432917 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:01.433000 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:01.433252 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:01.433254 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:01.433255 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:01.433389 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:01.433473 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:01.433627 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 22:32:01.433705 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 22:32:01.434620 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 8695
I20260808 22:32:01.446157 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 8901
I20260808 22:32:01.454756 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 9122
I20260808 22:32:01.464807 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 8565
2026-08-08T22:32:01Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2208 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:32:01 dist-test-slave-266d krb5kdc[9358](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:32:01 dist-test-slave-266d krb5kdc[9358](info): set up 2 sockets
Aug 08 22:32:01 dist-test-slave-266d krb5kdc[9358](info): commencing operation
krb5kdc: starting...
W20260808 22:32:02.621071 32094 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Aug 08 22:32:02 dist-test-slave-266d krb5kdc[9358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228322, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:32:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:32:02Z Disabled control of system clock
WARNING: no policy specified for oryx/127.31.87.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.87.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.87.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.190.keytab.
Entry for principal oryx/127.31.87.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.190.keytab.
I20260808 22:32:02.736181 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:41935
--webserver_interface=127.31.87.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.190.keytab
--principal=oryx/127.31.87.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:40667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:41935
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:32:02.843757 9380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:32:02.844002 9380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:32:02.844058 9380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:32:02.847988 9380 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:32:02.848043 9380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:32:02.848078 9380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:32:02.848101 9380 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:32:02.848129 9380 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:32:02.853173 9380 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:40667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:41935
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.190.keytab
--principal=oryx/127.31.87.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:41935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.87.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9380
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:32:02.854133 9380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:32:02.855079 9380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:32:02.863210 9389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:02.863394 9380 server_base.cc:1034] running on GCE node
W20260808 22:32:02.863226 9392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:02.863269 9390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:02.864063 9380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:32:02.865037 9380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:32:02.866231 9380 hybrid_clock.cc:648] HybridClock initialized: now 1786228322866217 us; error 25 us; skew 500 ppm
Aug 08 22:32:02 dist-test-slave-266d krb5kdc[9358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228322, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.87.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:32:02.868788 9380 init.cc:377] Logged in from keytab as oryx/127.31.87.190@KRBTEST.COM (short username oryx)
I20260808 22:32:02.869665 9380 webserver.cc:460] Webserver started at http://127.31.87.190:42311/ using document root <none> and password file <none>
I20260808 22:32:02.870162 9380 fs_manager.cc:359] Metadata directory not provided
I20260808 22:32:02.870217 9380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:32:02.870378 9380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:32:02.872304 9380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-0/data/instance:
uuid: "7ecff71ab1744a56b7959dc9072f4b64"
format_stamp: "Formatted at 2026-08-08 22:32:02 on dist-test-slave-266d"
server_key: "ce244a4c14d7dce3e884c0ed114a48ce"
server_key_iv: "903370493270f6688f1fed3bb5ff32bd"
server_key_version: "encryptionkey@0"
I20260808 22:32:02.872792 9380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-0/wal/instance:
uuid: "7ecff71ab1744a56b7959dc9072f4b64"
format_stamp: "Formatted at 2026-08-08 22:32:02 on dist-test-slave-266d"
server_key: "ce244a4c14d7dce3e884c0ed114a48ce"
server_key_iv: "903370493270f6688f1fed3bb5ff32bd"
server_key_version: "encryptionkey@0"
I20260808 22:32:02.876330 9380 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 22:32:02.878654 9403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:02.879863 9380 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260808 22:32:02.879989 9380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "7ecff71ab1744a56b7959dc9072f4b64"
format_stamp: "Formatted at 2026-08-08 22:32:02 on dist-test-slave-266d"
server_key: "ce244a4c14d7dce3e884c0ed114a48ce"
server_key_iv: "903370493270f6688f1fed3bb5ff32bd"
server_key_version: "encryptionkey@0"
I20260808 22:32:02.880085 9380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:32:02.890988 9380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:32:02.893812 9380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:32:02.893980 9380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:32:02.901286 9380 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.190:41935
I20260808 22:32:02.901546 9482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.190:41935 every 8 connection(s)
I20260808 22:32:02.902220 9380 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:32:02.904866 9483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:32:02.910218 9483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64: Bootstrap starting.
I20260808 22:32:02.911448 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 9380
I20260808 22:32:02.911583 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-0/wal/instance
I20260808 22:32:02.911837 32094 external_mini_cluster.cc:1428] Setting key e40e60663efdf6c9c2aeeac73b6062e4
I20260808 22:32:02.912781 9483 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64: Neither blocks nor log segments found. Creating new log.
I20260808 22:32:02.913530 9483 log.cc:824] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64: Log is configured to *not* fsync() on all Append() calls
I20260808 22:32:02.915513 9483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64: No bootstrap required, opened a new log
I20260808 22:32:02.919476 9483 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } }
I20260808 22:32:02.919754 9483 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:32:02.919816 9483 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ecff71ab1744a56b7959dc9072f4b64, State: Initialized, Role: FOLLOWER
Aug 08 22:32:02 dist-test-slave-266d krb5kdc[9358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228322, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM
I20260808 22:32:02.920295 9483 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } }
I20260808 22:32:02.920461 9483 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:32:02.920532 9483 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:32:02.920642 9483 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:32:02.921468 9483 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } }
I20260808 22:32:02.921778 9483 leader_election.cc:302] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7ecff71ab1744a56b7959dc9072f4b64; no voters:
I20260808 22:32:02.922056 9483 leader_election.cc:288] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:32:02.922169 9490 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:32:02.922367 9490 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 [term 1 LEADER]: Becoming Leader. State: Replica: 7ecff71ab1744a56b7959dc9072f4b64, State: Running, Role: LEADER
I20260808 22:32:02.922668 9490 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } }
I20260808 22:32:02.923250 9483 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:32:02.923792 9490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ecff71ab1744a56b7959dc9072f4b64. Latest consensus state: current_term: 1 leader_uuid: "7ecff71ab1744a56b7959dc9072f4b64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } }
I20260808 22:32:02.923914 9490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 [sys.catalog]: This master's current role is: LEADER
I20260808 22:32:02.924230 9492 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7ecff71ab1744a56b7959dc9072f4b64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } }
I20260808 22:32:02.924425 9492 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 [sys.catalog]: This master's current role is: LEADER
I20260808 22:32:02.926630 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:02.912891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46646 (local address 127.31.87.190:41935)
0808 22:32:02.913349 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:02.913364 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:02.913399 (+ 35us) server_negotiation.cc:407] Connection header received
0808 22:32:02.914031 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:02.914051 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:02.914351 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:02.914622 (+ 271us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:02.915464 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:02.916453 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:02.917628 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:02.918079 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:02.918190 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:02.920979 (+ 2789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:02.920984 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:02.920997 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:02.921022 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:02.922888 (+ 1866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:02.923423 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:02.923429 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:02.923434 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:02.923574 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:02.924046 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:02.924049 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:02.924051 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:02.924556 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:02.924745 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:02.925012 (+ 267us) server_negotiation.cc:299] Negotiation successful
0808 22:32:02.925575 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":171,"threads_started":1}
W20260808 22:32:02.928133 9515 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:32:02.928196 9515 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:32:02.928260 9495 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:32:02.929016 9495 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:32:02.934267 9495 catalog_manager.cc:1347] Generated new cluster ID: 50f463a19cad4af0a0f0aba6c2c7cfdd
I20260808 22:32:02.934316 9495 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:32:02.944144 9495 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:32:02.945310 9495 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:32:02.960057 9495 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64: Generated new TSK 0
I20260808 22:32:02.960829 9495 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.31.87.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.87.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.87.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.129.keytab.
Entry for principal oryx/127.31.87.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.129.keytab.
I20260808 22:32:02.991644 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.129:0
--local_ip_for_outbound_sockets=127.31.87.129
--webserver_interface=127.31.87.129
--webserver_port=0
--tserver_master_addrs=127.31.87.190:41935
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.129.keytab
--principal=oryx/127.31.87.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:40667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:32:03.099460 9522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:32:03.099699 9522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:32:03.099738 9522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:32:03.103611 9522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:32:03.103665 9522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:32:03.103732 9522 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.129
I20260808 22:32:03.108816 9522 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:40667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.129.keytab
--principal=oryx/127.31.87.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.87.129
--webserver_port=0
--tserver_master_addrs=127.31.87.190:41935
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9522
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.129
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:32:03.109830 9522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:32:03.110808 9522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 22:32:03.119856 9522 server_base.cc:1034] running on GCE node
W20260808 22:32:03.119901 9534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:03.119865 9529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:03.119846 9531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:03.120399 9522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:32:03.120985 9522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:32:03.122145 9522 hybrid_clock.cc:648] HybridClock initialized: now 1786228323122128 us; error 32 us; skew 500 ppm
Aug 08 22:32:03 dist-test-slave-266d krb5kdc[9358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228323, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.87.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:32:03.124900 9522 init.cc:377] Logged in from keytab as oryx/127.31.87.129@KRBTEST.COM (short username oryx)
I20260808 22:32:03.125773 9522 webserver.cc:460] Webserver started at http://127.31.87.129:37855/ using document root <none> and password file <none>
I20260808 22:32:03.126281 9522 fs_manager.cc:359] Metadata directory not provided
I20260808 22:32:03.126346 9522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:32:03.126528 9522 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:32:03.128451 9522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-0/data/instance:
uuid: "2d486908e4e54ccbb78867ae276bf39f"
format_stamp: "Formatted at 2026-08-08 22:32:03 on dist-test-slave-266d"
server_key: "71f07c2590a4877fef590ac413409eea"
server_key_iv: "375f938451460a76be5cae2229c8c503"
server_key_version: "encryptionkey@0"
I20260808 22:32:03.128978 9522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-0/wal/instance:
uuid: "2d486908e4e54ccbb78867ae276bf39f"
format_stamp: "Formatted at 2026-08-08 22:32:03 on dist-test-slave-266d"
server_key: "71f07c2590a4877fef590ac413409eea"
server_key_iv: "375f938451460a76be5cae2229c8c503"
server_key_version: "encryptionkey@0"
I20260808 22:32:03.132351 9522 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.002s
I20260808 22:32:03.134588 9542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:03.135694 9522 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260808 22:32:03.135795 9522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-0/wal
uuid: "2d486908e4e54ccbb78867ae276bf39f"
format_stamp: "Formatted at 2026-08-08 22:32:03 on dist-test-slave-266d"
server_key: "71f07c2590a4877fef590ac413409eea"
server_key_iv: "375f938451460a76be5cae2229c8c503"
server_key_version: "encryptionkey@0"
I20260808 22:32:03.135882 9522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:32:03.151576 9522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:32:03.154421 9522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:32:03.154578 9522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:32:03.155092 9522 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:32:03.155846 9522 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:32:03.155891 9522 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:03.155939 9522 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:32:03.155966 9522 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:03.164000 9522 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.129:44881
I20260808 22:32:03.164022 9725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.129:44881 every 8 connection(s)
I20260808 22:32:03.164783 9522 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
I20260808 22:32:03.167493 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 9522
I20260808 22:32:03.167619 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-0/wal/instance
I20260808 22:32:03.167898 32094 external_mini_cluster.cc:1428] Setting key 5bda560fba8ead55c57320ee396ab4c0
Aug 08 22:32:03 dist-test-slave-266d krb5kdc[9358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228323, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.87.129@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM
WARNING: no policy specified for oryx/127.31.87.130@KRBTEST.COM; defaulting to no policy
I20260808 22:32:03.178213 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:03.166474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:51055 (local address 127.31.87.190:41935)
0808 22:32:03.166646 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:03.166650 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:03.167404 (+ 754us) server_negotiation.cc:407] Connection header received
0808 22:32:03.168872 (+ 1468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:03.168876 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:03.168930 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:03.168998 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:03.170137 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:03.170649 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:03.172157 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:03.172411 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:03.172511 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:03.174779 (+ 2268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:03.174784 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:03.174787 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:03.174815 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:03.176324 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:03.176772 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:03.176776 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:03.176778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:03.176864 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:03.177173 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:03.177176 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:03.177178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:03.177397 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:03.177500 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:03.177955 (+ 455us) server_negotiation.cc:299] Negotiation successful
0808 22:32:03.178082 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 22:32:03.178951 9728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:03.166800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41935 (local address 127.31.87.129:51055)
0808 22:32:03.167242 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:03.167282 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:03.168669 (+ 1387us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:03.169194 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:03.169201 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:03.169539 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:03.169997 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:03.170010 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:03.170764 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:03.170767 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:03.172023 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:03.172036 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:03.172564 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:03.172567 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:03.172712 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:03.173245 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:03.173261 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:03.174619 (+ 1358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:32:03.176432 (+ 1813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:03.176436 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:03.176447 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:03.176679 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:03.176942 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:03.176944 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:03.176945 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:03.177058 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:03.177493 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:32:03.177498 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:32:03.177711 (+ 213us) client_negotiation.cc:769] Sending connection context
0808 22:32:03.177868 (+ 157us) client_negotiation.cc:240] Negotiation successful
0808 22:32:03.178130 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":116,"threads_started":1}
I20260808 22:32:03.180079 9726 heartbeater.cc:347] Connected to a master server at 127.31.87.190:41935
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.87.130@KRBTEST.COM" created.
I20260808 22:32:03.180317 9726 heartbeater.cc:464] Registering TS with master...
I20260808 22:32:03.180794 9726 heartbeater.cc:511] Master 127.31.87.190:41935 requested a full tablet report, sending...
I20260808 22:32:03.182178 9423 ts_manager.cc:194] Registered new tserver with Master: 2d486908e4e54ccbb78867ae276bf39f (127.31.87.129:44881)
I20260808 22:32:03.183286 9423 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.87.129@KRBTEST.COM'} at 127.31.87.129:51055
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.87.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.130.keytab.
Entry for principal oryx/127.31.87.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.130.keytab.
I20260808 22:32:03.193408 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.130:0
--local_ip_for_outbound_sockets=127.31.87.130
--webserver_interface=127.31.87.130
--webserver_port=0
--tserver_master_addrs=127.31.87.190:41935
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.130.keytab
--principal=oryx/127.31.87.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:40667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:32:03.298763 9732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:32:03.298987 9732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:32:03.299028 9732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:32:03.302915 9732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:32:03.302963 9732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:32:03.303031 9732 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.130
I20260808 22:32:03.307971 9732 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:40667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.130.keytab
--principal=oryx/127.31.87.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.87.130
--webserver_port=0
--tserver_master_addrs=127.31.87.190:41935
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9732
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.130
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:32:03.308866 9732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:32:03.309815 9732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:32:03.318854 9744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:03.318856 9740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:03.319034 9732 server_base.cc:1034] running on GCE node
W20260808 22:32:03.319248 9742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:03.319608 9732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:32:03.320179 9732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:32:03.321324 9732 hybrid_clock.cc:648] HybridClock initialized: now 1786228323321297 us; error 38 us; skew 500 ppm
Aug 08 22:32:03 dist-test-slave-266d krb5kdc[9358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228323, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.87.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:32:03.323904 9732 init.cc:377] Logged in from keytab as oryx/127.31.87.130@KRBTEST.COM (short username oryx)
I20260808 22:32:03.324759 9732 webserver.cc:460] Webserver started at http://127.31.87.130:35553/ using document root <none> and password file <none>
I20260808 22:32:03.325350 9732 fs_manager.cc:359] Metadata directory not provided
I20260808 22:32:03.325410 9732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:32:03.325592 9732 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:32:03.327731 9732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-1/data/instance:
uuid: "0490e64eb57d43268a8e1c61d601e835"
format_stamp: "Formatted at 2026-08-08 22:32:03 on dist-test-slave-266d"
server_key: "aa03647fec3dcca116cf9d00884be319"
server_key_iv: "c3df0566e6c3da3e68a9058b01d1c081"
server_key_version: "encryptionkey@0"
I20260808 22:32:03.328258 9732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-1/wal/instance:
uuid: "0490e64eb57d43268a8e1c61d601e835"
format_stamp: "Formatted at 2026-08-08 22:32:03 on dist-test-slave-266d"
server_key: "aa03647fec3dcca116cf9d00884be319"
server_key_iv: "c3df0566e6c3da3e68a9058b01d1c081"
server_key_version: "encryptionkey@0"
I20260808 22:32:03.331738 9732 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260808 22:32:03.333977 9755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:03.335078 9732 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:32:03.335186 9732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-1/wal
uuid: "0490e64eb57d43268a8e1c61d601e835"
format_stamp: "Formatted at 2026-08-08 22:32:03 on dist-test-slave-266d"
server_key: "aa03647fec3dcca116cf9d00884be319"
server_key_iv: "c3df0566e6c3da3e68a9058b01d1c081"
server_key_version: "encryptionkey@0"
I20260808 22:32:03.335280 9732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:32:03.353324 9732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:32:03.356139 9732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:32:03.356298 9732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:32:03.356827 9732 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:32:03.357767 9732 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:32:03.357825 9732 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:03.357885 9732 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:32:03.357921 9732 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:03.366555 9732 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.130:35045
I20260808 22:32:03.366572 9943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.130:35045 every 8 connection(s)
I20260808 22:32:03.367419 9732 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
I20260808 22:32:03.368719 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 9732
I20260808 22:32:03.368837 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-1/wal/instance
I20260808 22:32:03.369143 32094 external_mini_cluster.cc:1428] Setting key 80294e55c617e68b3ce5b72aa261c933
Aug 08 22:32:03 dist-test-slave-266d krb5kdc[9358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228323, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.87.130@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM
WARNING: no policy specified for oryx/127.31.87.131@KRBTEST.COM; defaulting to no policy
I20260808 22:32:03.381006 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:03.369174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:40493 (local address 127.31.87.190:41935)
0808 22:32:03.369320 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:03.369324 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:03.370158 (+ 834us) server_negotiation.cc:407] Connection header received
0808 22:32:03.371536 (+ 1378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:03.371540 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:03.371591 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:03.371675 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:03.372745 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:03.373537 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:03.374697 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:03.375030 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:03.375140 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:03.377655 (+ 2515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:03.377660 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:03.377663 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:03.377689 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:03.379241 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:03.379711 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:03.379714 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:03.379715 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:03.379797 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:03.380113 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:03.380115 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:03.380116 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:03.380277 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:03.380349 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:03.380797 (+ 448us) server_negotiation.cc:299] Negotiation successful
0808 22:32:03.380885 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.87.131@KRBTEST.COM" created.
I20260808 22:32:03.381929 9948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:03.369541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41935 (local address 127.31.87.130:40493)
0808 22:32:03.370002 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:03.370055 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:03.371328 (+ 1273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:03.371856 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:03.371862 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:03.372179 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:03.372606 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:03.372620 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:03.373675 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:03.373678 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:03.374540 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:03.374551 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:03.375175 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:03.375178 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:03.375319 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:03.375886 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:03.375903 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:03.377483 (+ 1580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:32:03.379350 (+ 1867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:03.379355 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:03.379367 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:03.379628 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:03.379863 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:03.379865 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:03.379866 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:03.380008 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:03.380364 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:32:03.380369 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:32:03.380595 (+ 226us) client_negotiation.cc:769] Sending connection context
0808 22:32:03.380760 (+ 165us) client_negotiation.cc:240] Negotiation successful
0808 22:32:03.380994 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":113,"threads_started":1}
I20260808 22:32:03.383033 9945 heartbeater.cc:347] Connected to a master server at 127.31.87.190:41935
I20260808 22:32:03.383298 9945 heartbeater.cc:464] Registering TS with master...
I20260808 22:32:03.383870 9945 heartbeater.cc:511] Master 127.31.87.190:41935 requested a full tablet report, sending...
I20260808 22:32:03.384979 9423 ts_manager.cc:194] Registered new tserver with Master: 0490e64eb57d43268a8e1c61d601e835 (127.31.87.130:35045)
I20260808 22:32:03.385545 9423 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.87.130@KRBTEST.COM'} at 127.31.87.130:40493
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.87.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.131.keytab.
Entry for principal oryx/127.31.87.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.131.keytab.
I20260808 22:32:03.396143 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.131:0
--local_ip_for_outbound_sockets=127.31.87.131
--webserver_interface=127.31.87.131
--webserver_port=0
--tserver_master_addrs=127.31.87.190:41935
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.131.keytab
--principal=oryx/127.31.87.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:40667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:32:03.501715 9951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:32:03.501952 9951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:32:03.502009 9951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:32:03.505870 9951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:32:03.505918 9951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:32:03.505991 9951 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.131
I20260808 22:32:03.510881 9951 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:40667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.131.keytab
--principal=oryx/127.31.87.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.87.131
--webserver_port=0
--tserver_master_addrs=127.31.87.190:41935
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9951
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.131
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:32:03.511795 9951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:32:03.512706 9951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:32:03.521720 9964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:03.521829 9951 server_base.cc:1034] running on GCE node
W20260808 22:32:03.521730 9963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:03.521734 9967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:03.522364 9951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:32:03.522944 9951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:32:03.524108 9951 hybrid_clock.cc:648] HybridClock initialized: now 1786228323524093 us; error 27 us; skew 500 ppm
Aug 08 22:32:03 dist-test-slave-266d krb5kdc[9358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228323, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.87.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:32:03.526760 9951 init.cc:377] Logged in from keytab as oryx/127.31.87.131@KRBTEST.COM (short username oryx)
I20260808 22:32:03.527524 9951 webserver.cc:460] Webserver started at http://127.31.87.131:37581/ using document root <none> and password file <none>
I20260808 22:32:03.528030 9951 fs_manager.cc:359] Metadata directory not provided
I20260808 22:32:03.528093 9951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:32:03.528273 9951 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:32:03.530174 9951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-2/data/instance:
uuid: "80ab594182ec4c6c95579360d4cc7210"
format_stamp: "Formatted at 2026-08-08 22:32:03 on dist-test-slave-266d"
server_key: "12aa4a892d5828f4c90f460c37f4f8b4"
server_key_iv: "de5d36bc1233b9767952b929be3ec814"
server_key_version: "encryptionkey@0"
I20260808 22:32:03.530682 9951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-2/wal/instance:
uuid: "80ab594182ec4c6c95579360d4cc7210"
format_stamp: "Formatted at 2026-08-08 22:32:03 on dist-test-slave-266d"
server_key: "12aa4a892d5828f4c90f460c37f4f8b4"
server_key_iv: "de5d36bc1233b9767952b929be3ec814"
server_key_version: "encryptionkey@0"
I20260808 22:32:03.534373 9951 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260808 22:32:03.536582 9975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:03.537699 9951 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:32:03.537806 9951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-2/wal
uuid: "80ab594182ec4c6c95579360d4cc7210"
format_stamp: "Formatted at 2026-08-08 22:32:03 on dist-test-slave-266d"
server_key: "12aa4a892d5828f4c90f460c37f4f8b4"
server_key_iv: "de5d36bc1233b9767952b929be3ec814"
server_key_version: "encryptionkey@0"
I20260808 22:32:03.537901 9951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:32:03.558607 9951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:32:03.561050 9951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:32:03.561229 9951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:32:03.561736 9951 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:32:03.562471 9951 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:32:03.562516 9951 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:03.562557 9951 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:32:03.562587 9951 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:03.570428 9951 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.131:46167
I20260808 22:32:03.570453 10145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.131:46167 every 8 connection(s)
I20260808 22:32:03.571187 9951 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
I20260808 22:32:03.571724 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 9951
I20260808 22:32:03.571892 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-2/wal/instance
I20260808 22:32:03.572185 32094 external_mini_cluster.cc:1428] Setting key 388060a3077202dee3256c261dded29e
Aug 08 22:32:03 dist-test-slave-266d krb5kdc[9358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228323, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.87.131@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM
I20260808 22:32:03.583490 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:03.572882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:52171 (local address 127.31.87.190:41935)
0808 22:32:03.572996 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:03.573000 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:03.573893 (+ 893us) server_negotiation.cc:407] Connection header received
0808 22:32:03.575232 (+ 1339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:03.575235 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:03.575299 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:03.575368 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:03.576439 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:03.576951 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:03.578065 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:03.578289 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:03.578376 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:03.580479 (+ 2103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:03.580482 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:03.580484 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:03.580505 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:03.581647 (+ 1142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:03.582108 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:03.582111 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:03.582112 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:03.582187 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:03.582482 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:03.582486 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:03.582487 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:03.582652 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:03.582735 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:03.583190 (+ 455us) server_negotiation.cc:299] Negotiation successful
0808 22:32:03.583335 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 22:32:03.584136 10149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:03.573288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41935 (local address 127.31.87.131:52171)
0808 22:32:03.573771 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:03.573809 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:03.575045 (+ 1236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:03.575539 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:03.575546 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:03.575873 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:03.576297 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:03.576322 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:03.577065 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:03.577068 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:03.577952 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:03.577962 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:03.578420 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:03.578424 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:03.578592 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:03.579029 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:03.579045 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:03.580330 (+ 1285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:32:03.581752 (+ 1422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:03.581772 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:03.581785 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:03.582015 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:03.582268 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:03.582271 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:03.582274 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:03.582395 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:03.582750 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:32:03.582765 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:32:03.582975 (+ 210us) client_negotiation.cc:769] Sending connection context
0808 22:32:03.583129 (+ 154us) client_negotiation.cc:240] Negotiation successful
0808 22:32:03.583416 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":127,"threads_started":1}
I20260808 22:32:03.585330 10147 heartbeater.cc:347] Connected to a master server at 127.31.87.190:41935
I20260808 22:32:03.585539 10147 heartbeater.cc:464] Registering TS with master...
I20260808 22:32:03.585975 10147 heartbeater.cc:511] Master 127.31.87.190:41935 requested a full tablet report, sending...
I20260808 22:32:03.586892 9423 ts_manager.cc:194] Registered new tserver with Master: 80ab594182ec4c6c95579360d4cc7210 (127.31.87.131:46167)
I20260808 22:32:03.587405 9423 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.87.131@KRBTEST.COM'} at 127.31.87.131:52171
I20260808 22:32:03.597265 32094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.31.87.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.87.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.87.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.189.keytab.
Entry for principal oryx/127.31.87.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.189.keytab.
I20260808 22:32:03.621328 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.189:34383
--webserver_interface=127.31.87.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.189.keytab
--principal=oryx/127.31.87.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:40667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:41935,127.31.87.189:34383
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:32:03.728536 10152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:32:03.728776 10152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:32:03.728828 10152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:32:03.732653 10152 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:32:03.732702 10152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:32:03.732729 10152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:32:03.732750 10152 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:32:03.732775 10152 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:32:03.737744 10152 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:40667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:41935,127.31.87.189:34383
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.189.keytab
--principal=oryx/127.31.87.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.189:34383
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.31.87.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10152
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
W20260808 22:32:03.737928 10152 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.31.87.190:41935,127.31.87.189:34383'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260808 22:32:03.738723 10152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:32:03.739665 10152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:32:03.747987 10160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:03.747989 10164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:03.748004 10162 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:03.748308 10152 server_base.cc:1034] running on GCE node
I20260808 22:32:03.748852 10152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:32:03.749939 10152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:32:03.751094 10152 hybrid_clock.cc:648] HybridClock initialized: now 1786228323751080 us; error 34 us; skew 500 ppm
Aug 08 22:32:03 dist-test-slave-266d krb5kdc[9358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228323, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.87.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:32:03.753785 10152 init.cc:377] Logged in from keytab as oryx/127.31.87.189@KRBTEST.COM (short username oryx)
I20260808 22:32:03.754565 10152 webserver.cc:460] Webserver started at http://127.31.87.189:45337/ using document root <none> and password file <none>
I20260808 22:32:03.755052 10152 fs_manager.cc:359] Metadata directory not provided
I20260808 22:32:03.755123 10152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:32:03.755312 10152 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:32:03.757280 10152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-1/data/instance:
uuid: "f9dc0652a85f416a9c61d82088095a21"
format_stamp: "Formatted at 2026-08-08 22:32:03 on dist-test-slave-266d"
server_key: "e95bb5ef85b83b24c001035dfe1aeb36"
server_key_iv: "a911848b5a85c0567466360f44415b8d"
server_key_version: "encryptionkey@0"
I20260808 22:32:03.757797 10152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-1/wal/instance:
uuid: "f9dc0652a85f416a9c61d82088095a21"
format_stamp: "Formatted at 2026-08-08 22:32:03 on dist-test-slave-266d"
server_key: "e95bb5ef85b83b24c001035dfe1aeb36"
server_key_iv: "a911848b5a85c0567466360f44415b8d"
server_key_version: "encryptionkey@0"
I20260808 22:32:03.761188 10152 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 22:32:03.763355 10177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:03.764456 10152 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:32:03.764561 10152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-1/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-1/wal
uuid: "f9dc0652a85f416a9c61d82088095a21"
format_stamp: "Formatted at 2026-08-08 22:32:03 on dist-test-slave-266d"
server_key: "e95bb5ef85b83b24c001035dfe1aeb36"
server_key_iv: "a911848b5a85c0567466360f44415b8d"
server_key_version: "encryptionkey@0"
I20260808 22:32:03.764647 10152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:32:03.775738 10152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:32:03.778319 10152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:32:03.778472 10152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:32:03.785255 10152 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.189:34383
I20260808 22:32:03.785511 10259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.189:34383 every 8 connection(s)
I20260808 22:32:03.786156 10152 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-1/data/info.pb
I20260808 22:32:03.786978 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 10152
I20260808 22:32:03.789276 10261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:32:03.793589 10261 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 22:32:03 dist-test-slave-266d krb5kdc[9358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228322, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.87.189@KRBTEST.COM
Aug 08 22:32:03 dist-test-slave-266d krb5kdc[9358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228323, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.87.189@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM
I20260808 22:32:03.801319 10265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:03.788423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40120 (local address 127.31.87.189:34383)
0808 22:32:03.788808 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:03.788826 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:03.788866 (+ 40us) server_negotiation.cc:407] Connection header received
0808 22:32:03.789691 (+ 825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:03.789715 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:03.790099 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:03.790445 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:03.791112 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:03.792590 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:03.793438 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:03.794103 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:03.794254 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:03.797050 (+ 2796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:03.797054 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:03.797062 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:03.797107 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:03.798583 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:03.798977 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:03.798984 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:03.798990 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:03.799128 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:03.799424 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:03.799427 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:03.799428 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:03.799772 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:03.799970 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:03.800190 (+ 220us) server_negotiation.cc:299] Negotiation successful
0808 22:32:03.800451 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":111,"threads_started":1}
I20260808 22:32:03.803917 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:03.795433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46662 (local address 127.31.87.190:41935)
0808 22:32:03.795546 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:03.795551 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:03.795846 (+ 295us) server_negotiation.cc:407] Connection header received
0808 22:32:03.796059 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:03.796062 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:03.796111 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:03.796179 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:03.796977 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:03.797504 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:03.798372 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:03.798581 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:03.798685 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:03.800870 (+ 2185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:03.800873 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:03.800875 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:03.800898 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:03.802350 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:03.802789 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:03.802791 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:03.802792 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:03.802856 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:03.803093 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:03.803094 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:03.803096 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:03.803336 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:03.803428 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:03.803716 (+ 288us) server_negotiation.cc:299] Negotiation successful
0808 22:32:03.803805 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 22:32:03.803920 10266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:03.795407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41935 (local address 127.0.0.1:46662)
0808 22:32:03.795723 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:03.795761 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:03.795938 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:03.796245 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:03.796251 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:03.796695 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:03.796869 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:03.796876 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:03.797612 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:03.797616 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:03.798256 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:03.798266 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:03.798698 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:03.798702 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:03.798812 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:03.799285 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:03.799299 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:03.800746 (+ 1447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:32:03.802441 (+ 1695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:03.802448 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:03.802452 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:03.802708 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:03.802925 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:03.802928 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:03.802930 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:03.803017 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:03.803441 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:32:03.803446 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:32:03.803553 (+ 107us) client_negotiation.cc:769] Sending connection context
0808 22:32:03.803661 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 22:32:03.803794 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":95,"threads_started":1}
I20260808 22:32:03.804966 10261 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.87.189" port: 34383 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 22:32:03 dist-test-slave-266d krb5kdc[9358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228323, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.87.189@KRBTEST.COM for oryx/127.31.87.189@KRBTEST.COM
I20260808 22:32:03.811712 10266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:03.805373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.189:34383 (local address 127.0.0.1:40130)
0808 22:32:03.805471 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:03.805484 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:03.805557 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:03.805731 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:03.805734 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:03.805873 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:03.805948 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:03.805953 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:03.806921 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:03.806925 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:03.807399 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:03.807406 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:03.807814 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:03.807817 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:03.807901 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:03.808249 (+ 348us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:03.808261 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:03.809293 (+ 1032us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:32:03.810522 (+ 1229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:03.810525 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:03.810527 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:03.810688 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:03.810902 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:03.810905 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:03.810907 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:03.810976 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:03.811269 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:32:03.811271 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:32:03.811347 (+ 76us) client_negotiation.cc:769] Sending connection context
0808 22:32:03.811431 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 22:32:03.811549 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 22:32:03.811726 10265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:03.805409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40130 (local address 127.31.87.189:34383)
0808 22:32:03.805488 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:03.805491 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:03.805523 (+ 32us) server_negotiation.cc:407] Connection header received
0808 22:32:03.805617 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:03.805619 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:03.805652 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:03.805711 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:03.806040 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:03.806814 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:03.807493 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:03.807723 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:03.807803 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:03.809401 (+ 1598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:03.809404 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:03.809406 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:03.809428 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:03.810434 (+ 1006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:03.810763 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:03.810766 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:03.810767 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:03.810830 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:03.811046 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:03.811049 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:03.811051 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:03.811192 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:03.811275 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:03.811479 (+ 204us) server_negotiation.cc:299] Negotiation successful
0808 22:32:03.811601 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":31}
I20260808 22:32:03.814179 10261 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21: Bootstrap starting.
I20260808 22:32:03.816263 10261 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21: Neither blocks nor log segments found. Creating new log.
I20260808 22:32:03.816946 10261 log.cc:824] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21: Log is configured to *not* fsync() on all Append() calls
I20260808 22:32:03.818658 10261 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21: No bootstrap required, opened a new log
I20260808 22:32:03.821521 10261 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 34383 } }
I20260808 22:32:03.821714 10261 raft_consensus.cc:374] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:32:03.821770 10261 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9dc0652a85f416a9c61d82088095a21, State: Initialized, Role: FOLLOWER
I20260808 22:32:03.822163 10261 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 34383 } }
I20260808 22:32:03.822564 10268 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 34383 } } }
I20260808 22:32:03.822671 10268 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:32:03.823292 10261 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 22:32:03.827823 10286 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:32:03.827903 10286 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:32:03.830041 10152 master_runner.cc:428] Detected that this master f9dc0652a85f416a9c61d82088095a21 is joining an existing cluster
I20260808 22:32:03.830087 10152 master_runner.cc:432] Initiating AddMaster RPC to add 127.31.87.189:34383
I20260808 22:32:03.841017 10265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:03.832282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40146 (local address 127.31.87.189:34383)
0808 22:32:03.832377 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:03.832381 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:03.832418 (+ 37us) server_negotiation.cc:407] Connection header received
0808 22:32:03.832454 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:03.832457 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:03.832499 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:03.832567 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:03.833229 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:03.833994 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:03.835007 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:03.835344 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:03.835446 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:03.836837 (+ 1391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:03.836841 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:03.836843 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:03.836869 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:03.838662 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:03.839878 (+ 1216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:03.839881 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:03.839883 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:03.839957 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:03.840195 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:03.840198 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:03.840200 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:03.840389 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:03.840454 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:03.840739 (+ 285us) server_negotiation.cc:299] Negotiation successful
0808 22:32:03.840859 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 22:32:03.841776 10295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:03.831898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.189:34383 (local address 127.0.0.1:40146)
0808 22:32:03.832249 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:03.832271 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:03.832363 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:03.832602 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:03.832605 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:03.832886 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:03.833105 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:03.833112 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:03.834100 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:03.834103 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:03.834907 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:03.834915 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:03.835447 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:03.835450 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:03.835524 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:03.836202 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:03.836219 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:03.836699 (+ 480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:32:03.839286 (+ 2587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:03.839289 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:03.839291 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:03.839776 (+ 485us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:03.840043 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:03.840046 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:03.840047 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:03.840118 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:03.840508 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:32:03.840511 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:32:03.840585 (+ 74us) client_negotiation.cc:769] Sending connection context
0808 22:32:03.841452 (+ 867us) client_negotiation.cc:240] Negotiation successful
0808 22:32:03.841583 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"mutex_wait_us":18,"spinlock_wait_cycles":3968,"thread_start_us":101,"threads_started":1}
I20260808 22:32:03.842025 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:03.831884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46668 (local address 127.31.87.190:41935)
0808 22:32:03.831999 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:03.832004 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:03.832347 (+ 343us) server_negotiation.cc:407] Connection header received
0808 22:32:03.832480 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:03.832483 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:03.832526 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:03.832589 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:03.833241 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:03.833833 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:03.834945 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:03.835198 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:03.835253 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:03.837412 (+ 2159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:03.837415 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:03.837417 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:03.837439 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:03.839154 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:03.839959 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:03.839963 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:03.839965 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:03.840079 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:03.840645 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:03.840649 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:03.840650 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:03.840939 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:03.841053 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:03.841656 (+ 603us) server_negotiation.cc:299] Negotiation successful
0808 22:32:03.841825 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 22:32:03.843531 10294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:03.831967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41935 (local address 127.0.0.1:46668)
0808 22:32:03.832272 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:03.832283 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:03.832397 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:03.832697 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:03.832701 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:03.832908 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:03.833081 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:03.833113 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:03.833932 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:03.833936 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:03.834812 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:03.834823 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:03.835398 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:03.835401 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:03.835487 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:03.836725 (+ 1238us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:03.836739 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:03.837051 (+ 312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:32:03.839275 (+ 2224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:03.839279 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:03.839281 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:03.839795 (+ 514us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:03.840248 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:03.840251 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:03.840253 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:03.840483 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:03.841182 (+ 699us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:32:03.841185 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:32:03.841339 (+ 154us) client_negotiation.cc:769] Sending connection context
0808 22:32:03.843195 (+ 1856us) client_negotiation.cc:240] Negotiation successful
0808 22:32:03.843340 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":30,"thread_start_us":90,"threads_started":1}
Aug 08 22:32:03 dist-test-slave-266d krb5kdc[9358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228322, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.87.190@KRBTEST.COM for oryx/127.31.87.189@KRBTEST.COM
I20260808 22:32:03.857118 10265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:03.848152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40152 (local address 127.31.87.189:34383)
0808 22:32:03.848256 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:03.848260 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:03.848646 (+ 386us) server_negotiation.cc:407] Connection header received
0808 22:32:03.848825 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:03.848827 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:03.848869 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:03.848928 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:03.849686 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:03.850144 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:03.851313 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:03.851498 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:03.851561 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:03.853811 (+ 2250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:03.853814 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:03.853816 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:03.853836 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:03.855308 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:03.855832 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:03.855836 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:03.855838 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:03.855921 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:03.856189 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:03.856191 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:03.856193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:03.856393 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:03.856482 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:03.856843 (+ 361us) server_negotiation.cc:299] Negotiation successful
0808 22:32:03.856950 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 22:32:03.857177 10300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:03.848100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.189:34383 (local address 127.0.0.1:40152)
0808 22:32:03.848501 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:03.848561 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:03.848704 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:03.848976 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:03.848981 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:03.849458 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:03.849586 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:03.849594 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:03.850232 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:03.850235 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:03.851215 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:03.851226 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:03.851580 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:03.851584 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:03.851691 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:03.852138 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:03.852154 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:03.853641 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:32:03.855419 (+ 1778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:03.855430 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:03.855436 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:03.855726 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:03.856002 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:03.856005 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:03.856006 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:03.856097 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:03.856490 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:32:03.856496 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:32:03.856623 (+ 127us) client_negotiation.cc:769] Sending connection context
0808 22:32:03.856775 (+ 152us) client_negotiation.cc:240] Negotiation successful
0808 22:32:03.856965 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":123,"threads_started":1}
I20260808 22:32:03.858696 9423 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.31.87.189:34383: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "f9dc0652a85f416a9c61d82088095a21"
member_type: NON_VOTER
last_known_addr {
host: "127.31.87.189"
port: 34383
}
attrs {
promote: true
}
}
dest_uuid: "7ecff71ab1744a56b7959dc9072f4b64"
cas_config_opid_index: -1
I20260808 22:32:03.859452 9423 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: NON_VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: true } }
I20260808 22:32:03.861402 10302 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER f9dc0652a85f416a9c61d82088095a21 (127.31.87.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: NON_VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: true } } }
I20260808 22:32:03.862165 10217 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:32:03.863348 10302 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.31.87.189:34383
WARNING: no policy specified for oryx/127.31.87.188@KRBTEST.COM; defaulting to no policy
I20260808 22:32:03.863515 10217 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21 [term 1 FOLLOWER]: Refusing update from remote peer 7ecff71ab1744a56b7959dc9072f4b64: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260808 22:32:03.863830 10302 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "7ecff71ab1744a56b7959dc9072f4b64" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: NON_VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: true } } }
I20260808 22:32:03.864001 10152 master.cc:537] Master@127.31.87.189:34383 shutting down...
I20260808 22:32:03.864009 10302 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 [sys.catalog]: This master's current role is: LEADER
I20260808 22:32:03.864295 10302 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: NON_VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 22:32:03.866514 9404 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260808 22:32:03.867403 9404 consensus_peers.cc:487] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 -> Peer f9dc0652a85f416a9c61d82088095a21 (127.31.87.189:34383): Couldn't send request to peer f9dc0652a85f416a9c61d82088095a21. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.87.188@KRBTEST.COM" created.
I20260808 22:32:03.868745 10152 raft_consensus.cc:2229] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:32:03.869025 10152 raft_consensus.cc:2258] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:32:03.869159 10152 tablet_replica.cc:330] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21: stopping tablet replica
I20260808 22:32:03.874289 10152 master.cc:552] Master@127.31.87.189:34383 shutdown complete.
I20260808 22:32:03.874339 10152 master_runner.cc:305] Clearing existing system tablet
I20260808 22:32:03.874565 10152 fs_manager.cc:359] Metadata directory not provided
I20260808 22:32:03.874658 10152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:32:03.875895 10152 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 22:32:03.876785 10313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:03.877008 10152 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:32:03.877169 10152 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:32:03.877211 10152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-1/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-1/wal
uuid: "f9dc0652a85f416a9c61d82088095a21"
format_stamp: "Formatted at 2026-08-08 22:32:03 on dist-test-slave-266d"
server_key: "e95bb5ef85b83b24c001035dfe1aeb36"
server_key_iv: "a911848b5a85c0567466360f44415b8d"
server_key_version: "encryptionkey@0"
I20260808 22:32:03.879287 10152 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:32:03.882393 10152 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 22:32:03.882530 10152 log.cc:1196] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.87.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.188.keytab.
Entry for principal oryx/127.31.87.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.188.keytab.
I20260808 22:32:03.882777 10152 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21: Deleting consensus metadata
I20260808 22:32:03.882969 10152 master_runner.cc:315] Copying system tablet from 127.31.87.190:41935
I20260808 22:32:03.883560 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.188:37451
--webserver_interface=127.31.87.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.188.keytab
--principal=oryx/127.31.87.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:40667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:41935,127.31.87.189:34383,127.31.87.188:37451
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 22:32:03.884243 10152 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21: tablet copy: Beginning tablet copy session from remote peer at address 127.31.87.190:41935
I20260808 22:32:03.893970 10324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:03.885422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41935 (local address 127.0.0.1:46674)
0808 22:32:03.886099 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:03.886111 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:03.886217 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:03.886463 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:03.886467 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:03.886666 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:03.886795 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:03.886802 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:03.887844 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:03.887848 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:03.888771 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:03.888782 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:03.889355 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:03.889359 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:03.889456 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:03.889991 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:03.890006 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:03.890440 (+ 434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:32:03.892305 (+ 1865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:03.892309 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:03.892312 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:03.892558 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:03.892877 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:03.892880 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:03.892882 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:03.892969 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:03.893428 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:32:03.893431 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:32:03.893529 (+ 98us) client_negotiation.cc:769] Sending connection context
0808 22:32:03.893629 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 22:32:03.893763 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":84,"threads_started":1}
I20260808 22:32:03.893976 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:03.885492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46674 (local address 127.31.87.190:41935)
0808 22:32:03.885612 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:03.885617 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:03.886188 (+ 571us) server_negotiation.cc:407] Connection header received
0808 22:32:03.886301 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:03.886305 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:03.886354 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:03.886429 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:03.886928 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:03.887730 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:03.888896 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:03.889238 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:03.889355 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:03.890587 (+ 1232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:03.890591 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:03.890594 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:03.890622 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:03.892195 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:03.892688 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:03.892692 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:03.892694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:03.892788 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:03.893060 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:03.893063 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:03.893065 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:03.893323 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:03.893422 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:03.893697 (+ 275us) server_negotiation.cc:299] Negotiation successful
0808 22:32:03.893817 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 22:32:03.894994 9459 tablet_copy_service.cc:140] P 7ecff71ab1744a56b7959dc9072f4b64: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer f9dc0652a85f416a9c61d82088095a21 ({username='oryx', principal='oryx/127.31.87.189@KRBTEST.COM'} at 127.0.0.1:46674)
I20260808 22:32:03.895192 9459 tablet_copy_service.cc:161] P 7ecff71ab1744a56b7959dc9072f4b64: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer f9dc0652a85f416a9c61d82088095a21 at {username='oryx', principal='oryx/127.31.87.189@KRBTEST.COM'} at 127.0.0.1:46674: session id = f9dc0652a85f416a9c61d82088095a21-00000000000000000000000000000000
I20260808 22:32:03.897171 9459 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 22:32:03.901428 10152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 22:32:03.903774 10152 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21: tablet copy: Starting download of 0 data blocks...
I20260808 22:32:03.903949 10152 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21: tablet copy: Starting download of 1 WAL segments...
I20260808 22:32:03.905845 10152 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 22:32:03.907467 9459 tablet_copy_service.cc:342] P 7ecff71ab1744a56b7959dc9072f4b64: Request end of tablet copy session f9dc0652a85f416a9c61d82088095a21-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.31.87.189@KRBTEST.COM'} at 127.0.0.1:46674
I20260808 22:32:03.907577 9459 tablet_copy_service.cc:433] P 7ecff71ab1744a56b7959dc9072f4b64: ending tablet copy session f9dc0652a85f416a9c61d82088095a21-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer f9dc0652a85f416a9c61d82088095a21
I20260808 22:32:03.914932 10152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:32:03.922202 10326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:03.922308 10328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:03.922237 10325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:03.929739 10152 server_base.cc:1034] running on GCE node
I20260808 22:32:03.929940 10152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:32:03.930248 10152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:32:03.931387 10152 hybrid_clock.cc:648] HybridClock initialized: now 1786228323931373 us; error 30 us; skew 500 ppm
Aug 08 22:32:03 dist-test-slave-266d krb5kdc[9358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228323, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.87.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:32:03.932984 10152 init.cc:377] Logged in from keytab as oryx/127.31.87.189@KRBTEST.COM (short username oryx)
I20260808 22:32:03.933450 10152 webserver.cc:460] Webserver started at http://127.31.87.189:43599/ using document root <none> and password file <none>
I20260808 22:32:03.933629 10152 fs_manager.cc:359] Metadata directory not provided
I20260808 22:32:03.933670 10152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:32:03.934934 10152 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 22:32:03.935889 10342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:03.936121 10152 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 22:32:03.936239 10152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-1/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-1/wal
uuid: "f9dc0652a85f416a9c61d82088095a21"
format_stamp: "Formatted at 2026-08-08 22:32:03 on dist-test-slave-266d"
server_key: "e95bb5ef85b83b24c001035dfe1aeb36"
server_key_iv: "a911848b5a85c0567466360f44415b8d"
server_key_version: "encryptionkey@0"
I20260808 22:32:03.936419 10152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:32:03.945492 10152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:32:03.947553 10152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:32:03.952523 10152 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.189:34383
I20260808 22:32:03.952567 10423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.189:34383 every 8 connection(s)
I20260808 22:32:03.953235 10152 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-1/data/info.pb
I20260808 22:32:03.955176 10426 sys_catalog.cc:263] Verifying existing consensus state
I20260808 22:32:03.956207 10426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21: Bootstrap starting.
I20260808 22:32:03.967378 10426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 22:32:03.967648 10426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21: Bootstrap complete.
I20260808 22:32:03.968233 10426 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: NON_VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: true } }
I20260808 22:32:03.968377 10426 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f9dc0652a85f416a9c61d82088095a21, State: Initialized, Role: LEARNER
I20260808 22:32:03.968497 10426 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: NON_VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: true } }
I20260808 22:32:03.968842 10429 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: NON_VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: true } } }
I20260808 22:32:03.968968 10426 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:32:03.969010 10429 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21 [sys.catalog]: This master's current role is: LEARNER
I20260808 22:32:03.972352 10448 catalog_manager.cc:1259] Loaded cluster ID: 50f463a19cad4af0a0f0aba6c2c7cfdd
I20260808 22:32:03.972407 10448 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21: loading cluster ID for follower catalog manager: success
I20260808 22:32:03.974493 10448 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21: acquiring CA information for follower catalog manager: success
I20260808 22:32:03.975813 10448 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 22:32:04.011790 10322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:32:04.012022 10322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:32:04.012066 10322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:32:04.015975 10322 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:32:04.016026 10322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:32:04.016044 10322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:32:04.016058 10322 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:32:04.016073 10322 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:32:04.021168 10322 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:40667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:41935,127.31.87.189:34383,127.31.87.188:37451
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.188.keytab
--principal=oryx/127.31.87.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.188:37451
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.31.87.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10322
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:32:04.022146 10322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:32:04.023141 10322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:32:04.031503 10459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:04.031503 10456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:04.031536 10457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:04.031761 10322 server_base.cc:1034] running on GCE node
I20260808 22:32:04.032296 10322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:32:04.033375 10322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:32:04.034534 10322 hybrid_clock.cc:648] HybridClock initialized: now 1786228324034510 us; error 31 us; skew 500 ppm
Aug 08 22:32:04 dist-test-slave-266d krb5kdc[9358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228324, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.87.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:32:04.037261 10322 init.cc:377] Logged in from keytab as oryx/127.31.87.188@KRBTEST.COM (short username oryx)
I20260808 22:32:04.038138 10322 webserver.cc:460] Webserver started at http://127.31.87.188:45201/ using document root <none> and password file <none>
I20260808 22:32:04.038695 10322 fs_manager.cc:359] Metadata directory not provided
I20260808 22:32:04.038789 10322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:32:04.039000 10322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:32:04.041083 10322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-2/data/instance:
uuid: "396475d452f242d59b60b887d7c332f0"
format_stamp: "Formatted at 2026-08-08 22:32:04 on dist-test-slave-266d"
server_key: "0fe7c54ca78070747cdeb3138b32eff0"
server_key_iv: "3c6119386ab25d863d956b8dedc648ce"
server_key_version: "encryptionkey@0"
I20260808 22:32:04.041616 10322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-2/wal/instance:
uuid: "396475d452f242d59b60b887d7c332f0"
format_stamp: "Formatted at 2026-08-08 22:32:04 on dist-test-slave-266d"
server_key: "0fe7c54ca78070747cdeb3138b32eff0"
server_key_iv: "3c6119386ab25d863d956b8dedc648ce"
server_key_version: "encryptionkey@0"
I20260808 22:32:04.045032 10322 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 22:32:04.047240 10467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:04.048368 10322 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 22:32:04.048471 10322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-2/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-2/wal
uuid: "396475d452f242d59b60b887d7c332f0"
format_stamp: "Formatted at 2026-08-08 22:32:04 on dist-test-slave-266d"
server_key: "0fe7c54ca78070747cdeb3138b32eff0"
server_key_iv: "3c6119386ab25d863d956b8dedc648ce"
server_key_version: "encryptionkey@0"
I20260808 22:32:04.048563 10322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:32:04.073686 10322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:32:04.076635 10322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:32:04.076797 10322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:32:04.083537 10322 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.188:37451
I20260808 22:32:04.083823 10551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.188:37451 every 8 connection(s)
I20260808 22:32:04.085763 10322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-2/data/info.pb
I20260808 22:32:04.088358 10552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:32:04.090341 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 10322
I20260808 22:32:04.092510 10552 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 22:32:04 dist-test-slave-266d krb5kdc[9358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228324, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.87.188@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM
Aug 08 22:32:04 dist-test-slave-266d krb5kdc[9358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228322, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.87.188@KRBTEST.COM
I20260808 22:32:04.103387 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:04.094680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46686 (local address 127.31.87.190:41935)
0808 22:32:04.094805 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:04.094809 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:04.095042 (+ 233us) server_negotiation.cc:407] Connection header received
0808 22:32:04.095173 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:04.095177 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:04.095223 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:04.095290 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:04.096063 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.096568 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.097498 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.097704 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.097799 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:04.099965 (+ 2166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:04.099969 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:04.099972 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:04.100000 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:04.101462 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:04.101946 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:04.101948 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:04.101949 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:04.102013 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:04.102327 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:04.102330 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:04.102332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:04.102514 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:04.102609 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:04.103122 (+ 513us) server_negotiation.cc:299] Negotiation successful
0808 22:32:04.103232 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 22:32:04.104399 10554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:04.091862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42096 (local address 127.31.87.188:37451)
0808 22:32:04.092273 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:04.092287 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:04.092320 (+ 33us) server_negotiation.cc:407] Connection header received
0808 22:32:04.092960 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:04.092977 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:04.093353 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:04.093687 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:04.094541 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.095966 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.096837 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.097330 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.097466 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:04.100424 (+ 2958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:04.100430 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:04.100442 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:04.100466 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:04.102033 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:04.102413 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:04.102417 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:04.102418 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:04.102483 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:04.102747 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:04.102750 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:04.102752 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:04.102996 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:04.103179 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:04.103366 (+ 187us) server_negotiation.cc:299] Negotiation successful
0808 22:32:04.103592 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":279,"thread_start_us":131,"threads_started":1}
I20260808 22:32:04.104406 10555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:04.094540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41935 (local address 127.0.0.1:46686)
0808 22:32:04.094917 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:04.094964 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:04.095084 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:04.095350 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:04.095355 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:04.095746 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:04.095958 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:04.095965 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.096695 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.096698 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:04.097386 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:04.097396 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.097850 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.097853 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:04.097941 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:04.098329 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:04.098350 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:04.099825 (+ 1475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:32:04.101559 (+ 1734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:04.101566 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:04.101575 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:04.101862 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:04.102086 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:04.102089 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:04.102091 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:04.102228 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:04.102611 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:32:04.102616 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:32:04.102855 (+ 239us) client_negotiation.cc:769] Sending connection context
0808 22:32:04.103047 (+ 192us) client_negotiation.cc:240] Negotiation successful
0808 22:32:04.103334 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":32,"thread_start_us":129,"threads_started":1}
I20260808 22:32:04.105867 10552 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.87.189" port: 34383 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 22:32:04 dist-test-slave-266d krb5kdc[9358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228324, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.87.188@KRBTEST.COM for oryx/127.31.87.189@KRBTEST.COM
I20260808 22:32:04.112739 10555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:04.106273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.189:34383 (local address 127.0.0.1:40154)
0808 22:32:04.106368 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:04.106382 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:04.106440 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:04.106806 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:04.106809 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:04.106921 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:04.107003 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:04.107008 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.107785 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.107787 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:04.108276 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:04.108282 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.108670 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.108672 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:04.108733 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:04.109061 (+ 328us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:04.109071 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:04.110126 (+ 1055us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:32:04.111745 (+ 1619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:04.111747 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:04.111748 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:04.111901 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:04.112108 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:04.112110 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:04.112111 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:04.112168 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:04.112422 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:32:04.112424 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:32:04.112487 (+ 63us) client_negotiation.cc:769] Sending connection context
0808 22:32:04.112551 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 22:32:04.112635 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 22:32:04.112799 10558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:04.106382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40154 (local address 127.31.87.189:34383)
0808 22:32:04.106620 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:04.106623 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:04.106634 (+ 11us) server_negotiation.cc:407] Connection header received
0808 22:32:04.106668 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:04.106670 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:04.106706 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:04.106772 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:04.107087 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.107711 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.108371 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.108589 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.108667 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:04.110266 (+ 1599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:04.110270 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:04.110272 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:04.110295 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:04.111666 (+ 1371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:04.111964 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:04.111967 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:04.111968 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:04.112035 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:04.112217 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:04.112218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:04.112219 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:04.112365 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:04.112429 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:04.112592 (+ 163us) server_negotiation.cc:299] Negotiation successful
0808 22:32:04.112679 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":90,"threads_started":1}
I20260808 22:32:04.113425 10552 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37451 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 22:32:04 dist-test-slave-266d krb5kdc[9358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228324, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.87.188@KRBTEST.COM for oryx/127.31.87.188@KRBTEST.COM
I20260808 22:32:04.119978 10555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:04.113793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.188:37451 (local address 127.0.0.1:42108)
0808 22:32:04.113885 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:04.113896 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:04.113953 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:04.114134 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:04.114136 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:04.114250 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:04.114312 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:04.114316 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.115320 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.115323 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:04.115798 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:04.115806 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.116178 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.116181 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:04.116252 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:04.116625 (+ 373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:04.116635 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:04.117686 (+ 1051us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:32:04.118949 (+ 1263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:04.118951 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:04.118953 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:04.119118 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:04.119316 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:04.119318 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:04.119319 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:04.119379 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:04.119632 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:32:04.119634 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:32:04.119696 (+ 62us) client_negotiation.cc:769] Sending connection context
0808 22:32:04.119770 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 22:32:04.119858 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 22:32:04.120051 10554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:04.113895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42108 (local address 127.31.87.188:37451)
0808 22:32:04.113991 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:04.113994 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:04.114004 (+ 10us) server_negotiation.cc:407] Connection header received
0808 22:32:04.114028 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:04.114030 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:04.114061 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:04.114116 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:04.114386 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.114835 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.115884 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.116083 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.116147 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:04.117793 (+ 1646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:04.117796 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:04.117798 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:04.117818 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:04.118857 (+ 1039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:04.119189 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:04.119191 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:04.119192 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:04.119255 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:04.119437 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:04.119439 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:04.119440 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:04.119570 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:04.119638 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:04.119794 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 22:32:04.119908 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":36}
I20260808 22:32:04.122478 10552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0: Bootstrap starting.
I20260808 22:32:04.124745 10552 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0: Neither blocks nor log segments found. Creating new log.
I20260808 22:32:04.125547 10552 log.cc:824] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0: Log is configured to *not* fsync() on all Append() calls
I20260808 22:32:04.127312 10552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0: No bootstrap required, opened a new log
I20260808 22:32:04.130172 10552 raft_consensus.cc:348] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 34383 } } peers { permanent_uuid: "396475d452f242d59b60b887d7c332f0" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37451 } }
I20260808 22:32:04.130371 10552 raft_consensus.cc:374] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:32:04.130427 10552 raft_consensus.cc:729] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 396475d452f242d59b60b887d7c332f0, State: Initialized, Role: FOLLOWER
I20260808 22:32:04.130815 10552 consensus_queue.cc:260] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 34383 } } peers { permanent_uuid: "396475d452f242d59b60b887d7c332f0" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37451 } }
I20260808 22:32:04.131220 10563 sys_catalog.cc:455] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 34383 } } peers { permanent_uuid: "396475d452f242d59b60b887d7c332f0" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37451 } } }
I20260808 22:32:04.131341 10563 sys_catalog.cc:458] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:32:04.131899 10552 sys_catalog.cc:564] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 22:32:04.136288 10579 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:32:04.136364 10579 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:32:04.139298 10322 master_runner.cc:428] Detected that this master 396475d452f242d59b60b887d7c332f0 is joining an existing cluster
I20260808 22:32:04.139358 10322 master_runner.cc:432] Initiating AddMaster RPC to add 127.31.87.188:37451
I20260808 22:32:04.149667 10591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:04.141886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41935 (local address 127.0.0.1:46690)
0808 22:32:04.142155 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:04.142169 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:04.142302 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:04.142653 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:04.142656 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:04.142864 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:04.143006 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:04.143013 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.144090 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.144093 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:04.145296 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:04.145308 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.145785 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.145788 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:04.145884 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:04.146260 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:04.146274 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:04.146733 (+ 459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:32:04.148299 (+ 1566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:04.148303 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:04.148305 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:04.148546 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:04.148760 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:04.148763 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:04.148766 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:04.148851 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:04.149161 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:32:04.149164 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:32:04.149255 (+ 91us) client_negotiation.cc:769] Sending connection context
0808 22:32:04.149358 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 22:32:04.149480 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":23,"thread_start_us":107,"threads_started":1}
I20260808 22:32:04.149688 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:04.141854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46690 (local address 127.31.87.190:41935)
0808 22:32:04.141975 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:04.141979 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:04.142235 (+ 256us) server_negotiation.cc:407] Connection header received
0808 22:32:04.142412 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:04.142416 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:04.142466 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:04.142516 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:04.143156 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.143946 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.145413 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.145675 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.145752 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:04.146864 (+ 1112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:04.146866 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:04.146868 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:04.146890 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:04.148200 (+ 1310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:04.148627 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:04.148629 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:04.148630 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:04.148693 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:04.148920 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:04.148921 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:04.148922 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:04.149066 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:04.149177 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:04.149427 (+ 250us) server_negotiation.cc:299] Negotiation successful
0808 22:32:04.149544 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 22:32:04.157455 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 9522
Aug 08 22:32:04 dist-test-slave-266d krb5kdc[9358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228322, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.87.190@KRBTEST.COM for oryx/127.31.87.188@KRBTEST.COM
I20260808 22:32:04.163358 10592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:04.142548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.189:34383 (local address 127.0.0.1:40156)
0808 22:32:04.149290 (+ 6742us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:04.149301 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:04.149422 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:04.149753 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:04.149755 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:04.149953 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:04.150056 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:04.150064 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.151046 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.151049 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:04.151915 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:04.151925 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.152522 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.152526 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:04.152608 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:04.152969 (+ 361us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:04.152983 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:04.153377 (+ 394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:32:04.155269 (+ 1892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:04.155272 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:04.155274 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:04.161795 (+ 6521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:04.162100 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:04.162102 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:04.162103 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:04.162167 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:04.162563 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:32:04.162566 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:32:04.162659 (+ 93us) client_negotiation.cc:769] Sending connection context
0808 22:32:04.163040 (+ 381us) client_negotiation.cc:240] Negotiation successful
0808 22:32:04.163165 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6667,"thread_start_us":74,"threads_started":1}
I20260808 22:32:04.163373 10558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:04.141877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40156 (local address 127.31.87.189:34383)
0808 22:32:04.142003 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:04.142007 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:04.149361 (+ 7354us) server_negotiation.cc:407] Connection header received
0808 22:32:04.149522 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:04.149525 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:04.149572 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:04.149620 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:04.150175 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.150935 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.152127 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.152347 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.152435 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:04.153525 (+ 1090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:04.153529 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:04.153531 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:04.153556 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:04.155152 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:04.161911 (+ 6759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:04.161915 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:04.161917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:04.162004 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:04.162261 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:04.162272 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:04.162274 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:04.162437 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:04.162518 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:04.163127 (+ 609us) server_negotiation.cc:299] Negotiation successful
0808 22:32:04.163245 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 22:32:04.166352 10554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:04.142323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42124 (local address 127.31.87.188:37451)
0808 22:32:04.142448 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:04.142451 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:04.142874 (+ 423us) server_negotiation.cc:407] Connection header received
0808 22:32:04.142965 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:04.142968 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:04.143003 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:04.143066 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:04.143455 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.143902 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.144690 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.144896 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.144976 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:04.161485 (+ 16509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:04.161489 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:04.161492 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:04.161525 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:04.164728 (+ 3203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:04.165249 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:04.165253 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:04.165254 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:04.165328 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:04.165583 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:04.165586 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:04.165588 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:04.165776 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:04.165837 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:04.166114 (+ 277us) server_negotiation.cc:299] Negotiation successful
0808 22:32:04.166210 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68,"spinlock_wait_cycles":3328}
I20260808 22:32:04.166776 10300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:04.153412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.188:37451 (local address 127.0.0.1:42126)
0808 22:32:04.153560 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:04.153572 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:04.153635 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:04.154043 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:04.154047 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:04.154215 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:04.154311 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:04.154318 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.155301 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.155305 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:04.156147 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:04.156158 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.156651 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.156654 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:04.156746 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:04.157196 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:04.157209 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:04.158383 (+ 1174us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:32:04.165304 (+ 6921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:04.165308 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:04.165310 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:04.165537 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:04.165830 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:04.165833 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:04.165834 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:04.165922 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:04.166317 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:32:04.166321 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:32:04.166418 (+ 97us) client_negotiation.cc:769] Sending connection context
0808 22:32:04.166501 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 22:32:04.166634 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89}
I20260808 22:32:04.166800 10593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:04.142292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.188:37451 (local address 127.0.0.1:42124)
0808 22:32:04.142794 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:04.142805 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:04.142891 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:04.143100 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:04.143103 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:04.143271 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:04.143359 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:04.143364 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.143990 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.143993 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:04.144593 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:04.144601 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.144993 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.144996 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:04.145074 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:04.145881 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:04.145891 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:04.161372 (+ 15481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:32:04.164818 (+ 3446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:04.164821 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:04.164823 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:04.165153 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:04.165421 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:04.165424 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:04.165426 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:04.165500 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:04.165890 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:32:04.165892 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:32:04.165970 (+ 78us) client_negotiation.cc:769] Sending connection context
0808 22:32:04.166508 (+ 538us) client_negotiation.cc:240] Negotiation successful
0808 22:32:04.166621 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"mutex_wait_us":38,"thread_start_us":85,"threads_started":1}
I20260808 22:32:04.167663 10594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:04.153470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42126 (local address 127.31.87.188:37451)
0808 22:32:04.153810 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:04.153817 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:04.153829 (+ 12us) server_negotiation.cc:407] Connection header received
0808 22:32:04.153886 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:04.153889 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:04.153932 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:04.154011 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:04.154441 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.155193 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.156246 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.156514 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.156640 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:04.161485 (+ 4845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:04.161489 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:04.161492 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:04.161527 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:04.165186 (+ 3659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:04.165644 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:04.165647 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:04.165648 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:04.165727 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:04.166026 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:04.166029 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:04.166031 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:04.166200 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:04.167342 (+ 1142us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:04.167410 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 22:32:04.167513 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"spinlock_wait_cycles":7168,"thread_start_us":101,"threads_started":1}
I20260808 22:32:04.168542 9423 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.31.87.188:37451: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "396475d452f242d59b60b887d7c332f0"
member_type: NON_VOTER
last_known_addr {
host: "127.31.87.188"
port: 37451
}
attrs {
promote: true
}
}
dest_uuid: "7ecff71ab1744a56b7959dc9072f4b64"
cas_config_opid_index: 5
I20260808 22:32:04.169138 9423 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: NON_VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: true } } peers { permanent_uuid: "396475d452f242d59b60b887d7c332f0" member_type: NON_VOTER last_known_addr { host: "127.31.87.188" port: 37451 } attrs { promote: true } }
I20260808 22:32:04.170655 10504 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:32:04.172096 10504 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0 [term 1 FOLLOWER]: Refusing update from remote peer 7ecff71ab1744a56b7959dc9072f4b64: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 22:32:04.172736 10302 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "396475d452f242d59b60b887d7c332f0" member_type: NON_VOTER last_known_addr { host: "127.31.87.188" port: 37451 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:32:04.174830 10300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:04.170219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.189:34383 (local address 127.0.0.1:40158)
0808 22:32:04.170530 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:04.170544 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:04.170611 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:04.171147 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:04.171150 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:04.171166 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:32:04.171236 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:04.171242 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.172162 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.172166 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:04.173304 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:04.173313 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.174345 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.174349 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:04.174432 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:04.174476 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 22:32:04.174546 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 22:32:04.174706 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242}
I20260808 22:32:04.175243 10558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:04.170415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40158 (local address 127.31.87.189:34383)
0808 22:32:04.170747 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:04.170751 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:04.170773 (+ 22us) server_negotiation.cc:407] Connection header received
0808 22:32:04.170812 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:04.170815 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:04.170869 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:04.171042 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:32:04.171353 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.172035 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.173435 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.174036 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.174136 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:04.174281 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:04.175073 (+ 792us) server_negotiation.cc:299] Negotiation successful
0808 22:32:04.175143 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264}
I20260808 22:32:04.175626 10374 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21 [term 1 LEARNER]: Refusing update from remote peer 7ecff71ab1744a56b7959dc9072f4b64: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 22:32:04.176054 10302 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: NON_VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260808 22:32:04.177814 10504 raft_consensus.cc:858] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 34383 } } peers { permanent_uuid: "396475d452f242d59b60b887d7c332f0" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37451 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: NON_VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: NON_VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: true } } peers { permanent_uuid: "396475d452f242d59b60b887d7c332f0" member_type: NON_VOTER last_known_addr { host: "127.31.87.188" port: 37451 } attrs { promote: true } }
W20260808 22:32:04.177990 10504 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 34383 } } peers { permanent_uuid: "396475d452f242d59b60b887d7c332f0" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37451 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: NON_VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: NON_VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: true } } peers { permanent_uuid: "396475d452f242d59b60b887d7c332f0" member_type: NON_VOTER last_known_addr { host: "127.31.87.188" port: 37451 } attrs { promote: true } }
I20260808 22:32:04.178158 10429 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ecff71ab1744a56b7959dc9072f4b64. Latest consensus state: current_term: 1 leader_uuid: "7ecff71ab1744a56b7959dc9072f4b64" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: NON_VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: true } } } pending_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: NON_VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: true } } peers { permanent_uuid: "396475d452f242d59b60b887d7c332f0" member_type: NON_VOTER last_known_addr { host: "127.31.87.188" port: 37451 } attrs { promote: true } } }
I20260808 22:32:04.178292 10429 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21 [sys.catalog]: This master's current role is: LEARNER
I20260808 22:32:04.180398 10504 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER f9dc0652a85f416a9c61d82088095a21 (127.31.87.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: NON_VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: true } } }
I20260808 22:32:04.182646 10600 sys_catalog.cc:455] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7ecff71ab1744a56b7959dc9072f4b64" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: NON_VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: true } } }
I20260808 22:32:04.182755 10600 sys_catalog.cc:458] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260808 22:32:04.182847 10600 sys_catalog.cc:455] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7ecff71ab1744a56b7959dc9072f4b64" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: NON_VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: true } } }
I20260808 22:32:04.182919 10600 sys_catalog.cc:458] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260808 22:32:04.183084 10563 sys_catalog.cc:455] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ecff71ab1744a56b7959dc9072f4b64. Latest consensus state: current_term: 1 leader_uuid: "7ecff71ab1744a56b7959dc9072f4b64" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: NON_VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: true } } }
I20260808 22:32:04.183144 10563 sys_catalog.cc:458] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260808 22:32:04.183951 10302 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7ecff71ab1744a56b7959dc9072f4b64" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: NON_VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: true } } } pending_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: NON_VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: true } } peers { permanent_uuid: "396475d452f242d59b60b887d7c332f0" member_type: NON_VOTER last_known_addr { host: "127.31.87.188" port: 37451 } attrs { promote: true } } }
I20260808 22:32:04.184088 10302 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 [sys.catalog]: This master's current role is: LEADER
I20260808 22:32:04.184767 10504 raft_consensus.cc:1467] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0 [term 1 NON_PARTICIPANT]: Allowing update even though not a member of the config
I20260808 22:32:04.184782 10302 raft_consensus.cc:1013] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 [term 1 LEADER]: attempt to promote peer f9dc0652a85f416a9c61d82088095a21: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20260808 22:32:04.185644 10302 raft_consensus.cc:1013] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 [term 1 LEADER]: attempt to promote peer 396475d452f242d59b60b887d7c332f0: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20260808 22:32:04.188803 10595 mvcc.cc:205] Tried to move back new op lower bound from 7316391210840772608 to 7316391210692157440. Current Snapshot: MvccSnapshot[applied={T|T < 7316391210775785472}]
I20260808 22:32:04.198340 10305 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 396475d452f242d59b60b887d7c332f0 (127.31.87.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: NON_VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: true } } peers { permanent_uuid: "396475d452f242d59b60b887d7c332f0" member_type: NON_VOTER last_known_addr { host: "127.31.87.188" port: 37451 } attrs { promote: true } } }
I20260808 22:32:04.199937 10305 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.31.87.188:37451
I20260808 22:32:04.200412 10322 master.cc:537] Master@127.31.87.188:37451 shutting down...
I20260808 22:32:04.200582 10305 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7ecff71ab1744a56b7959dc9072f4b64" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: NON_VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: true } } peers { permanent_uuid: "396475d452f242d59b60b887d7c332f0" member_type: NON_VOTER last_known_addr { host: "127.31.87.188" port: 37451 } attrs { promote: true } } }
I20260808 22:32:04.200680 10305 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 [sys.catalog]: This master's current role is: LEADER
I20260808 22:32:04.201443 10302 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "7ecff71ab1744a56b7959dc9072f4b64" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: NON_VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: true } } peers { permanent_uuid: "396475d452f242d59b60b887d7c332f0" member_type: NON_VOTER last_known_addr { host: "127.31.87.188" port: 37451 } attrs { promote: true } } }
I20260808 22:32:04.201545 10302 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 [sys.catalog]: This master's current role is: LEADER
I20260808 22:32:04.202908 10374 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 396475d452f242d59b60b887d7c332f0 (127.31.87.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: NON_VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: true } } peers { permanent_uuid: "396475d452f242d59b60b887d7c332f0" member_type: NON_VOTER last_known_addr { host: "127.31.87.188" port: 37451 } attrs { promote: true } } }
I20260808 22:32:04.204697 10429 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7ecff71ab1744a56b7959dc9072f4b64" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: NON_VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: true } } peers { permanent_uuid: "396475d452f242d59b60b887d7c332f0" member_type: NON_VOTER last_known_addr { host: "127.31.87.188" port: 37451 } attrs { promote: true } } }
I20260808 22:32:04.204819 10429 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21 [sys.catalog]: This master's current role is: LEARNER
I20260808 22:32:04.205372 10322 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0 [term 1 LEARNER]: Raft consensus shutting down.
I20260808 22:32:04.205639 10322 pending_rounds.cc:62] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0: Trying to abort 1 pending ops.
W20260808 22:32:04.205659 9409 consensus_peers.cc:487] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 -> Peer 396475d452f242d59b60b887d7c332f0 (127.31.87.188:37451): Couldn't send request to peer 396475d452f242d59b60b887d7c332f0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 22:32:04.205719 10322 pending_rounds.cc:68] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0: Aborting op as it isn't in flight: id { term: 1 index: 6 } timestamp: 7316391215795048448 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "00000000000000000000000000000000" old_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: NON_VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: true } } } new_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: NON_VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: true } } peers { permanent_uuid: "396475d452f242d59b60b887d7c332f0" member_type: NON_VOTER last_known_addr { host: "127.31.87.188" port: 37451 } attrs { promote: true } } } }
I20260808 22:32:04.205855 10322 raft_consensus.cc:2894] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0 [term 1 LEARNER]: Aborting config change with OpId 1.6: Aborted: Op aborted
I20260808 22:32:04.205916 10322 raft_consensus.cc:2866] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0 [term 1 NON_PARTICIPANT]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260808 22:32:04.205965 10322 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260808 22:32:04.206036 10322 tablet_replica.cc:330] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0: stopping tablet replica
I20260808 22:32:04.206346 10302 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64: attempting to promote NON_VOTER f9dc0652a85f416a9c61d82088095a21 to VOTER
I20260808 22:32:04.206755 10302 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "396475d452f242d59b60b887d7c332f0" member_type: NON_VOTER last_known_addr { host: "127.31.87.188" port: 37451 } attrs { promote: true } }
I20260808 22:32:04.207288 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.129:44881
--local_ip_for_outbound_sockets=127.31.87.129
--tserver_master_addrs=127.31.87.190:41935
--webserver_port=37855
--webserver_interface=127.31.87.129
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.129.keytab
--principal=oryx/127.31.87.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:40667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.31.87.190:41935,127.31.87.189:34383
--tserver_master_addrs=127.31.87.190:41935,127.31.87.189:34383,127.31.87.188:37451 with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 22:32:04.208880 10374 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21 [term 1 LEARNER]: Refusing update from remote peer 7ecff71ab1744a56b7959dc9072f4b64: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20260808 22:32:04.209522 9409 consensus_peers.cc:487] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 -> Peer 396475d452f242d59b60b887d7c332f0 (127.31.87.188:37451): Couldn't send request to peer 396475d452f242d59b60b887d7c332f0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 22:32:04.209702 10608 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260808 22:32:04.211647 10305 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, f9dc0652a85f416a9c61d82088095a21 (127.31.87.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "396475d452f242d59b60b887d7c332f0" member_type: NON_VOTER last_known_addr { host: "127.31.87.188" port: 37451 } attrs { promote: true } } }
I20260808 22:32:04.212826 10305 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "7ecff71ab1744a56b7959dc9072f4b64" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "396475d452f242d59b60b887d7c332f0" member_type: NON_VOTER last_known_addr { host: "127.31.87.188" port: 37451 } attrs { promote: true } } }
I20260808 22:32:04.212927 10305 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 [sys.catalog]: This master's current role is: LEADER
I20260808 22:32:04.213222 10302 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7ecff71ab1744a56b7959dc9072f4b64" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "396475d452f242d59b60b887d7c332f0" member_type: NON_VOTER last_known_addr { host: "127.31.87.188" port: 37451 } attrs { promote: true } } }
I20260808 22:32:04.213331 10302 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 [sys.catalog]: This master's current role is: LEADER
I20260808 22:32:04.213550 10374 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, f9dc0652a85f416a9c61d82088095a21 (127.31.87.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "396475d452f242d59b60b887d7c332f0" member_type: NON_VOTER last_known_addr { host: "127.31.87.188" port: 37451 } attrs { promote: true } } }
I20260808 22:32:04.214708 10429 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7ecff71ab1744a56b7959dc9072f4b64" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "396475d452f242d59b60b887d7c332f0" member_type: NON_VOTER last_known_addr { host: "127.31.87.188" port: 37451 } attrs { promote: true } } }
I20260808 22:32:04.214813 10429 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:32:04.226647 10322 master.cc:552] Master@127.31.87.188:37451 shutdown complete.
I20260808 22:32:04.226701 10322 master_runner.cc:305] Clearing existing system tablet
I20260808 22:32:04.226941 10322 fs_manager.cc:359] Metadata directory not provided
I20260808 22:32:04.226994 10322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:32:04.228274 10322 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:32:04.229192 10616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:04.229417 10322 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:32:04.229543 10322 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:32:04.229576 10322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-2/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-2/wal
uuid: "396475d452f242d59b60b887d7c332f0"
format_stamp: "Formatted at 2026-08-08 22:32:04 on dist-test-slave-266d"
server_key: "0fe7c54ca78070747cdeb3138b32eff0"
server_key_iv: "3c6119386ab25d863d956b8dedc648ce"
server_key_version: "encryptionkey@0"
I20260808 22:32:04.231415 10322 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:32:04.233839 10322 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 22:32:04.233928 10322 log.cc:1196] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260808 22:32:04.234197 10322 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0: Deleting consensus metadata
I20260808 22:32:04.234350 10322 master_runner.cc:315] Copying system tablet from 127.31.87.190:41935
I20260808 22:32:04.235453 10322 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0: tablet copy: Beginning tablet copy session from remote peer at address 127.31.87.190:41935
I20260808 22:32:04.244849 10626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:04.236025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41935 (local address 127.0.0.1:46700)
0808 22:32:04.236409 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:04.236424 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:04.236539 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:04.236799 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:04.236802 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:04.237019 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:04.237511 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:04.237523 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.238600 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.238604 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:04.239565 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:04.239577 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.240154 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.240158 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:04.240255 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:04.240828 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:04.240843 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:04.241290 (+ 447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:32:04.243305 (+ 2015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:04.243309 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:04.243311 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:04.243557 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:04.243849 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:04.243852 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:04.243854 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:04.243943 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:04.244325 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:32:04.244329 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:32:04.244426 (+ 97us) client_negotiation.cc:769] Sending connection context
0808 22:32:04.244528 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 22:32:04.244665 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":88,"threads_started":1}
I20260808 22:32:04.244864 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:04.236078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46700 (local address 127.31.87.190:41935)
0808 22:32:04.236233 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:04.236238 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:04.236495 (+ 257us) server_negotiation.cc:407] Connection header received
0808 22:32:04.236633 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:04.236636 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:04.236690 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:04.236783 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:04.237658 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.238490 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.239701 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.240035 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.240141 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:04.241442 (+ 1301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:04.241446 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:04.241449 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:04.241476 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:04.243194 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:04.243661 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:04.243664 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:04.243666 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:04.243760 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:04.244027 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:04.244030 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:04.244032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:04.244233 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:04.244330 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:04.244591 (+ 261us) server_negotiation.cc:299] Negotiation successful
0808 22:32:04.244705 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 22:32:04.245725 9459 tablet_copy_service.cc:140] P 7ecff71ab1744a56b7959dc9072f4b64: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 396475d452f242d59b60b887d7c332f0 ({username='oryx', principal='oryx/127.31.87.188@KRBTEST.COM'} at 127.0.0.1:46700)
I20260808 22:32:04.245836 9459 tablet_copy_service.cc:161] P 7ecff71ab1744a56b7959dc9072f4b64: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 396475d452f242d59b60b887d7c332f0 at {username='oryx', principal='oryx/127.31.87.188@KRBTEST.COM'} at 127.0.0.1:46700: session id = 396475d452f242d59b60b887d7c332f0-00000000000000000000000000000000
I20260808 22:32:04.246883 9459 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 22:32:04.248070 10322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 22:32:04.250326 10322 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0: tablet copy: Starting download of 0 data blocks...
I20260808 22:32:04.250504 10322 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0: tablet copy: Starting download of 1 WAL segments...
I20260808 22:32:04.251904 10322 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 22:32:04.253623 9459 tablet_copy_service.cc:342] P 7ecff71ab1744a56b7959dc9072f4b64: Request end of tablet copy session 396475d452f242d59b60b887d7c332f0-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.31.87.188@KRBTEST.COM'} at 127.0.0.1:46700
I20260808 22:32:04.253706 9459 tablet_copy_service.cc:433] P 7ecff71ab1744a56b7959dc9072f4b64: ending tablet copy session 396475d452f242d59b60b887d7c332f0-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 396475d452f242d59b60b887d7c332f0
I20260808 22:32:04.260752 10322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:32:04.262389 10627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:04.262617 10632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:04.262638 10628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:04.262913 10322 server_base.cc:1034] running on GCE node
I20260808 22:32:04.263083 10322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:32:04.263372 10322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:32:04.264523 10322 hybrid_clock.cc:648] HybridClock initialized: now 1786228324264502 us; error 42 us; skew 500 ppm
Aug 08 22:32:04 dist-test-slave-266d krb5kdc[9358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228324, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.87.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:32:04.266197 10322 init.cc:377] Logged in from keytab as oryx/127.31.87.188@KRBTEST.COM (short username oryx)
I20260808 22:32:04.266636 10322 webserver.cc:460] Webserver started at http://127.31.87.188:46055/ using document root <none> and password file <none>
I20260808 22:32:04.266824 10322 fs_manager.cc:359] Metadata directory not provided
I20260808 22:32:04.266865 10322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:32:04.268098 10322 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:32:04.269117 10641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:04.269364 10322 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 22:32:04.269435 10322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-2/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-2/wal
uuid: "396475d452f242d59b60b887d7c332f0"
format_stamp: "Formatted at 2026-08-08 22:32:04 on dist-test-slave-266d"
server_key: "0fe7c54ca78070747cdeb3138b32eff0"
server_key_iv: "3c6119386ab25d863d956b8dedc648ce"
server_key_version: "encryptionkey@0"
I20260808 22:32:04.269508 10322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:32:04.277407 10322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:32:04.279464 10322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:32:04.284278 10322 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.188:37451
I20260808 22:32:04.284317 10725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.188:37451 every 8 connection(s)
I20260808 22:32:04.284962 10322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/master-2/data/info.pb
I20260808 22:32:04.286993 10726 sys_catalog.cc:263] Verifying existing consensus state
I20260808 22:32:04.288045 10726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0: Bootstrap starting.
I20260808 22:32:04.295688 10726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 22:32:04.295955 10726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0: Bootstrap complete.
I20260808 22:32:04.296509 10726 raft_consensus.cc:348] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "396475d452f242d59b60b887d7c332f0" member_type: NON_VOTER last_known_addr { host: "127.31.87.188" port: 37451 } attrs { promote: true } }
I20260808 22:32:04.296644 10726 raft_consensus.cc:729] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 396475d452f242d59b60b887d7c332f0, State: Initialized, Role: LEARNER
I20260808 22:32:04.296758 10726 consensus_queue.cc:260] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "396475d452f242d59b60b887d7c332f0" member_type: NON_VOTER last_known_addr { host: "127.31.87.188" port: 37451 } attrs { promote: true } }
I20260808 22:32:04.297076 10734 sys_catalog.cc:455] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "396475d452f242d59b60b887d7c332f0" member_type: NON_VOTER last_known_addr { host: "127.31.87.188" port: 37451 } attrs { promote: true } } }
I20260808 22:32:04.297221 10726 sys_catalog.cc:564] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:32:04.297291 10734 sys_catalog.cc:458] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0 [sys.catalog]: This master's current role is: LEARNER
I20260808 22:32:04.300915 10748 catalog_manager.cc:1259] Loaded cluster ID: 50f463a19cad4af0a0f0aba6c2c7cfdd
I20260808 22:32:04.300967 10748 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0: loading cluster ID for follower catalog manager: success
I20260808 22:32:04.302979 10748 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0: acquiring CA information for follower catalog manager: success
I20260808 22:32:04.304279 10748 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 22:32:04.341763 10612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:32:04.341979 10612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:32:04.342025 10612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:32:04.345897 10612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:32:04.345948 10612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:32:04.346019 10612 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.129
I20260808 22:32:04.351018 10612 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:40667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.129.keytab
--principal=oryx/127.31.87.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.129:44881
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.87.129
--webserver_port=37855
--tserver_master_addrs=127.31.87.190:41935,127.31.87.189:34383,127.31.87.188:37451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10612
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.129
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:32:04.351949 10612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:32:04.352936 10612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:32:04.362120 10762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:04.362118 10756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:04.362294 10612 server_base.cc:1034] running on GCE node
W20260808 22:32:04.362143 10754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:04.362701 10612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:32:04.363277 10612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:32:04.364427 10612 hybrid_clock.cc:648] HybridClock initialized: now 1786228324364402 us; error 38 us; skew 500 ppm
Aug 08 22:32:04 dist-test-slave-266d krb5kdc[9358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228324, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.87.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:32:04.366968 10612 init.cc:377] Logged in from keytab as oryx/127.31.87.129@KRBTEST.COM (short username oryx)
I20260808 22:32:04.367750 10612 webserver.cc:460] Webserver started at http://127.31.87.129:37855/ using document root <none> and password file <none>
I20260808 22:32:04.368256 10612 fs_manager.cc:359] Metadata directory not provided
I20260808 22:32:04.368328 10612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:32:04.372247 10612 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:32:04.374205 10776 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:04.375314 10612 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:32:04.375416 10612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-0/wal
uuid: "2d486908e4e54ccbb78867ae276bf39f"
format_stamp: "Formatted at 2026-08-08 22:32:03 on dist-test-slave-266d"
server_key: "71f07c2590a4877fef590ac413409eea"
server_key_iv: "375f938451460a76be5cae2229c8c503"
server_key_version: "encryptionkey@0"
I20260808 22:32:04.375892 10612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:32:04.387336 9945 heartbeater.cc:503] Master 127.31.87.190:41935 was elected leader, sending a full tablet report...
I20260808 22:32:04.388089 10612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:32:04.390956 10612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:32:04.391114 10612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:32:04.391629 10612 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:32:04.392379 10612 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:32:04.392423 10612 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:04.392462 10612 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:32:04.392489 10612 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:04.401948 10612 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.129:44881
I20260808 22:32:04.402010 10964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.129:44881 every 8 connection(s)
I20260808 22:32:04.402858 10612 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
I20260808 22:32:04.404558 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 10612
I20260808 22:32:04.404764 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 9732
Aug 08 22:32:04 dist-test-slave-266d krb5kdc[9358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228324, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.87.129@KRBTEST.COM for oryx/127.31.87.189@KRBTEST.COM
Aug 08 22:32:04 dist-test-slave-266d krb5kdc[9358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228324, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.87.129@KRBTEST.COM for oryx/127.31.87.188@KRBTEST.COM
Aug 08 22:32:04 dist-test-slave-266d krb5kdc[9358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228324, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.87.129@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM
I20260808 22:32:04.422520 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:04.404483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:44479 (local address 127.31.87.190:41935)
0808 22:32:04.404615 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:04.404621 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:04.407621 (+ 3000us) server_negotiation.cc:407] Connection header received
0808 22:32:04.408596 (+ 975us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:04.408610 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:04.408673 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:04.408770 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:04.410888 (+ 2118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.411888 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.414826 (+ 2938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.415221 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.415300 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:04.418503 (+ 3203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:04.418507 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:04.418510 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:04.418543 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:04.420587 (+ 2044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:04.421075 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:04.421080 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:04.421081 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:04.421197 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:04.421531 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:04.421534 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:04.421536 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:04.421770 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:04.421854 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:04.422220 (+ 366us) server_negotiation.cc:299] Negotiation successful
0808 22:32:04.422343 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 22:32:04.423067 10558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:04.406171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:58819 (local address 127.31.87.189:34383)
0808 22:32:04.406315 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:04.406321 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:04.409144 (+ 2823us) server_negotiation.cc:407] Connection header received
0808 22:32:04.409241 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:04.409245 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:04.409305 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:04.409387 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:04.410874 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.411711 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.413602 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.413955 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.414062 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:04.417822 (+ 3760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:04.417827 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:04.417830 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:04.417862 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:04.419617 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:04.420293 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:04.420298 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:04.420300 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:04.420393 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:04.420808 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:04.420812 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:04.420814 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:04.421045 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:04.421176 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:04.422788 (+ 1612us) server_negotiation.cc:299] Negotiation successful
0808 22:32:04.422901 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 22:32:04.423233 10973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:04.405151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.188:37451 (local address 127.31.87.129:58975)
0808 22:32:04.408745 (+ 3594us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:04.408758 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:04.408829 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:04.409653 (+ 824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:04.409656 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:04.409910 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:04.410692 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:04.410714 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.412082 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.412085 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:04.413447 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:04.413459 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.414147 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.414152 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:04.414381 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:04.415501 (+ 1120us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:04.415526 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:04.417865 (+ 2339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:32:04.420402 (+ 2537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:04.420406 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:04.420409 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:04.420709 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:04.421132 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:04.421136 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:04.421139 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:04.421252 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:04.421803 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:32:04.421807 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:32:04.421918 (+ 111us) client_negotiation.cc:769] Sending connection context
0808 22:32:04.422025 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 22:32:04.422181 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3534,"mutex_wait_us":242,"spinlock_wait_cycles":17024,"thread_start_us":2031,"threads_started":1}
I20260808 22:32:04.423327 10972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:04.404934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.189:34383 (local address 127.31.87.129:58819)
0808 22:32:04.409030 (+ 4096us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:04.409041 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:04.409153 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:04.409518 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:04.409521 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:04.409886 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:04.410692 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:04.410714 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.411892 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.411896 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:04.413447 (+ 1551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:04.413459 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.414193 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.414197 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:04.414380 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:04.415552 (+ 1172us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:04.415576 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:04.417594 (+ 2018us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:32:04.419761 (+ 2167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:04.419768 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:04.419784 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:04.420159 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:04.420509 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:04.420512 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:04.420514 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:04.420694 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:04.421238 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:32:04.421245 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:32:04.421543 (+ 298us) client_negotiation.cc:769] Sending connection context
0808 22:32:04.421741 (+ 198us) client_negotiation.cc:240] Negotiation successful
0808 22:32:04.422079 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4009,"mutex_wait_us":1654,"spinlock_wait_cycles":20736,"thread_start_us":833,"threads_started":1}
I20260808 22:32:04.423451 10969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:04.404931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41935 (local address 127.31.87.129:44479)
0808 22:32:04.407446 (+ 2515us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:04.407500 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:04.408392 (+ 892us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:04.409145 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:04.409158 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:04.409891 (+ 733us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:04.410690 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:04.410715 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.413551 (+ 2836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.413555 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:04.414656 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:04.414669 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.415474 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.415478 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:04.415583 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:04.416548 (+ 965us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:04.416566 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:04.418345 (+ 1779us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:32:04.420711 (+ 2366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:04.420714 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:04.420716 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:04.420954 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:04.421315 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:04.421318 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:04.421320 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:04.421413 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:04.421919 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:32:04.421922 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:32:04.422009 (+ 87us) client_negotiation.cc:769] Sending connection context
0808 22:32:04.422686 (+ 677us) client_negotiation.cc:240] Negotiation successful
0808 22:32:04.422861 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2303,"mutex_wait_us":61,"spinlock_wait_cycles":2816,"thread_start_us":148,"threads_started":1}
I20260808 22:32:04.424280 10974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:04.407511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:58975 (local address 127.31.87.188:37451)
0808 22:32:04.409376 (+ 1865us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:04.409391 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:04.409408 (+ 17us) server_negotiation.cc:407] Connection header received
0808 22:32:04.409455 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:04.409458 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:04.409509 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:04.409802 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:04.410904 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.411939 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.413603 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.413958 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.414779 (+ 821us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:04.418054 (+ 3275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:04.418060 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:04.418063 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:04.418093 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:04.420249 (+ 2156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:04.420850 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:04.420854 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:04.420856 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:04.420963 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:04.421385 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:04.421389 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:04.421391 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:04.421663 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:04.423791 (+ 2128us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:04.423889 (+ 98us) server_negotiation.cc:299] Negotiation successful
0808 22:32:04.424097 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1783,"thread_start_us":1332,"threads_started":1}
I20260808 22:32:04.424590 10967 heartbeater.cc:347] Connected to a master server at 127.31.87.189:34383
I20260808 22:32:04.424877 10967 heartbeater.cc:464] Registering TS with master...
I20260808 22:32:04.424952 10965 heartbeater.cc:347] Connected to a master server at 127.31.87.188:37451
I20260808 22:32:04.425041 10965 heartbeater.cc:464] Registering TS with master...
I20260808 22:32:04.425446 10965 heartbeater.cc:511] Master 127.31.87.188:37451 requested a full tablet report, sending...
I20260808 22:32:04.425453 10967 heartbeater.cc:511] Master 127.31.87.189:34383 requested a full tablet report, sending...
I20260808 22:32:04.426928 10360 ts_manager.cc:194] Registered new tserver with Master: 2d486908e4e54ccbb78867ae276bf39f (127.31.87.129:44881)
I20260808 22:32:04.426929 10667 ts_manager.cc:194] Registered new tserver with Master: 2d486908e4e54ccbb78867ae276bf39f (127.31.87.129:44881)
I20260808 22:32:04.437670 10966 heartbeater.cc:347] Connected to a master server at 127.31.87.190:41935
I20260808 22:32:04.437763 10966 heartbeater.cc:464] Registering TS with master...
I20260808 22:32:04.437953 10966 heartbeater.cc:511] Master 127.31.87.190:41935 requested a full tablet report, sending...
I20260808 22:32:04.438740 9423 ts_manager.cc:194] Re-registered known tserver with Master: 2d486908e4e54ccbb78867ae276bf39f (127.31.87.129:44881)
I20260808 22:32:04.439323 9423 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.87.129@KRBTEST.COM'} at 127.31.87.129:44479
I20260808 22:32:04.443703 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.130:35045
--local_ip_for_outbound_sockets=127.31.87.130
--tserver_master_addrs=127.31.87.190:41935
--webserver_port=35553
--webserver_interface=127.31.87.130
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.130.keytab
--principal=oryx/127.31.87.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:40667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.31.87.190:41935,127.31.87.189:34383
--tserver_master_addrs=127.31.87.190:41935,127.31.87.189:34383,127.31.87.188:37451 with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:32:04.551102 10975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:32:04.551316 10975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:32:04.551353 10975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:32:04.555164 10975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:32:04.555210 10975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:32:04.555262 10975 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.130
I20260808 22:32:04.560138 10975 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:40667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.130.keytab
--principal=oryx/127.31.87.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.130:35045
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.87.130
--webserver_port=35553
--tserver_master_addrs=127.31.87.190:41935,127.31.87.189:34383,127.31.87.188:37451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10975
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.130
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:32:04.560966 10975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:32:04.561862 10975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:32:04.570776 10986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:04.570750 10984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:04.570983 10975 server_base.cc:1034] running on GCE node
W20260808 22:32:04.570752 10981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:04.571569 10975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:32:04.572256 10975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:32:04.573438 10975 hybrid_clock.cc:648] HybridClock initialized: now 1786228324573410 us; error 34 us; skew 500 ppm
Aug 08 22:32:04 dist-test-slave-266d krb5kdc[9358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228324, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.87.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:32:04.576085 10975 init.cc:377] Logged in from keytab as oryx/127.31.87.130@KRBTEST.COM (short username oryx)
I20260808 22:32:04.576865 10975 webserver.cc:460] Webserver started at http://127.31.87.130:35553/ using document root <none> and password file <none>
I20260808 22:32:04.577409 10975 fs_manager.cc:359] Metadata directory not provided
I20260808 22:32:04.577484 10975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:32:04.580992 10975 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260808 22:32:04.582931 10997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:04.584085 10975 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:32:04.584188 10975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-1/wal
uuid: "0490e64eb57d43268a8e1c61d601e835"
format_stamp: "Formatted at 2026-08-08 22:32:03 on dist-test-slave-266d"
server_key: "aa03647fec3dcca116cf9d00884be319"
server_key_iv: "c3df0566e6c3da3e68a9058b01d1c081"
server_key_version: "encryptionkey@0"
I20260808 22:32:04.584682 10975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:32:04.589076 10147 heartbeater.cc:503] Master 127.31.87.190:41935 was elected leader, sending a full tablet report...
I20260808 22:32:04.598906 10975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:32:04.601320 10975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:32:04.601454 10975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:32:04.601917 10975 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:32:04.602650 10975 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:32:04.602695 10975 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:04.602730 10975 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:32:04.602751 10975 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:04.612002 10975 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.130:35045
I20260808 22:32:04.612020 11180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.130:35045 every 8 connection(s)
I20260808 22:32:04.612906 10975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
I20260808 22:32:04.619474 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 10975
I20260808 22:32:04.619665 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 9951
Aug 08 22:32:04 dist-test-slave-266d krb5kdc[9358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228324, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.87.130@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM
Aug 08 22:32:04 dist-test-slave-266d krb5kdc[9358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228324, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.87.130@KRBTEST.COM for oryx/127.31.87.189@KRBTEST.COM
Aug 08 22:32:04 dist-test-slave-266d krb5kdc[9358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228324, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.87.130@KRBTEST.COM for oryx/127.31.87.188@KRBTEST.COM
I20260808 22:32:04.629207 10974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:04.615223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:33255 (local address 127.31.87.188:37451)
0808 22:32:04.615350 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:04.615355 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:04.616225 (+ 870us) server_negotiation.cc:407] Connection header received
0808 22:32:04.616813 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:04.616817 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:04.616883 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:04.616955 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:04.618292 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.619191 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.620560 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.620989 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.621113 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:04.625212 (+ 4099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:04.625217 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:04.625220 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:04.625255 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:04.627105 (+ 1850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:04.627896 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:04.627899 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:04.627901 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:04.627985 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:04.628276 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:04.628279 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:04.628281 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:04.628491 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:04.628556 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:04.628879 (+ 323us) server_negotiation.cc:299] Negotiation successful
0808 22:32:04.629013 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 22:32:04.629217 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:04.614682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:54513 (local address 127.31.87.190:41935)
0808 22:32:04.614951 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:04.614956 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:04.616187 (+ 1231us) server_negotiation.cc:407] Connection header received
0808 22:32:04.616833 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:04.616837 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:04.616890 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:04.616941 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:04.618339 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.619109 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.620506 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.620848 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.620921 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:04.625074 (+ 4153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:04.625078 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:04.625081 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:04.625129 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:04.626690 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:04.627261 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:04.627276 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:04.627278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:04.627367 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:04.627725 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:04.627728 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:04.627730 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:04.627941 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:04.628041 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:04.628843 (+ 802us) server_negotiation.cc:299] Negotiation successful
0808 22:32:04.628993 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185}
I20260808 22:32:04.629789 10558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:04.614818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:37373 (local address 127.31.87.189:34383)
0808 22:32:04.615041 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:04.615045 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:04.615756 (+ 711us) server_negotiation.cc:407] Connection header received
0808 22:32:04.616837 (+ 1081us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:04.616840 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:04.616890 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:04.616941 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:04.618311 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.618845 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.620507 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.620852 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.620921 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:04.624743 (+ 3822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:04.624747 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:04.624751 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:04.624789 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:04.627241 (+ 2452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:04.628196 (+ 955us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:04.628199 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:04.628201 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:04.628282 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:04.628795 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:04.628799 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:04.628801 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:04.629014 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:04.629083 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:04.629524 (+ 441us) server_negotiation.cc:299] Negotiation successful
0808 22:32:04.629638 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152}
I20260808 22:32:04.630669 11196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:04.615068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.188:37451 (local address 127.31.87.130:33255)
0808 22:32:04.616119 (+ 1051us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:04.616132 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:04.616601 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:04.617163 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:04.617172 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:04.617638 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:04.618147 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:04.618161 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.619317 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.619321 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:04.620404 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:04.620417 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.621324 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.621329 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:04.621501 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:04.622183 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:04.622199 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:04.624973 (+ 2774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:32:04.627292 (+ 2319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:04.627296 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:04.627299 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:04.627789 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:04.628085 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:04.628088 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:04.628090 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:04.628169 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:04.628623 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:32:04.628625 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:32:04.628705 (+ 80us) client_negotiation.cc:769] Sending connection context
0808 22:32:04.629384 (+ 679us) client_negotiation.cc:240] Negotiation successful
0808 22:32:04.629691 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":960,"mutex_wait_us":311,"thread_start_us":108,"threads_started":1}
I20260808 22:32:04.630676 11189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:04.615068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41935 (local address 127.31.87.130:54513)
0808 22:32:04.616076 (+ 1008us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:04.616089 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:04.616603 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:04.617163 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:04.617172 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:04.617639 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:04.618148 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:04.618163 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.619226 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.619229 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:04.620375 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:04.620382 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.621134 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.621138 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:04.621431 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:04.622127 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:04.622149 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:04.624914 (+ 2765us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:32:04.626818 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:04.626824 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:04.626839 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:04.627146 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:04.627474 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:04.627476 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:04.627478 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:04.627640 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:04.628062 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:32:04.628069 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:32:04.628481 (+ 412us) client_negotiation.cc:769] Sending connection context
0808 22:32:04.630065 (+ 1584us) client_negotiation.cc:240] Negotiation successful
0808 22:32:04.630250 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":926,"mutex_wait_us":228,"spinlock_wait_cycles":13824,"thread_start_us":136,"threads_started":1}
I20260808 22:32:04.630890 11191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:04.615068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.189:34383 (local address 127.31.87.130:37373)
0808 22:32:04.615613 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:04.615661 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:04.616599 (+ 938us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:04.617163 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:04.617172 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:04.617631 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:04.618154 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:04.618163 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.618995 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.618997 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:04.620370 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:04.620381 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.621134 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.621138 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:04.621416 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:04.622286 (+ 870us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:04.622300 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:04.624499 (+ 2199us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:32:04.627359 (+ 2860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:04.627362 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:04.627363 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:04.628093 (+ 730us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:04.628589 (+ 496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:04.628592 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:04.628594 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:04.628676 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:04.629179 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:32:04.629182 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:32:04.629266 (+ 84us) client_negotiation.cc:769] Sending connection context
0808 22:32:04.629975 (+ 709us) client_negotiation.cc:240] Negotiation successful
0808 22:32:04.630102 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"mutex_wait_us":229,"spinlock_wait_cycles":40320,"thread_start_us":217,"threads_started":1}
I20260808 22:32:04.631990 11187 heartbeater.cc:347] Connected to a master server at 127.31.87.189:34383
I20260808 22:32:04.632014 11181 heartbeater.cc:347] Connected to a master server at 127.31.87.188:37451
I20260808 22:32:04.632226 11187 heartbeater.cc:464] Registering TS with master...
I20260808 22:32:04.632231 11181 heartbeater.cc:464] Registering TS with master...
I20260808 22:32:04.632714 11187 heartbeater.cc:511] Master 127.31.87.189:34383 requested a full tablet report, sending...
I20260808 22:32:04.632716 11181 heartbeater.cc:511] Master 127.31.87.188:37451 requested a full tablet report, sending...
I20260808 22:32:04.634047 10667 ts_manager.cc:194] Registered new tserver with Master: 0490e64eb57d43268a8e1c61d601e835 (127.31.87.130:35045)
I20260808 22:32:04.642093 11183 heartbeater.cc:347] Connected to a master server at 127.31.87.190:41935
I20260808 22:32:04.642145 10360 ts_manager.cc:194] Registered new tserver with Master: 0490e64eb57d43268a8e1c61d601e835 (127.31.87.130:35045)
I20260808 22:32:04.642170 11183 heartbeater.cc:464] Registering TS with master...
I20260808 22:32:04.642342 11183 heartbeater.cc:511] Master 127.31.87.190:41935 requested a full tablet report, sending...
I20260808 22:32:04.642906 9423 ts_manager.cc:194] Re-registered known tserver with Master: 0490e64eb57d43268a8e1c61d601e835 (127.31.87.130:35045)
I20260808 22:32:04.643486 9423 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.87.130@KRBTEST.COM'} at 127.31.87.130:54513
I20260808 22:32:04.648964 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.131:46167
--local_ip_for_outbound_sockets=127.31.87.131
--tserver_master_addrs=127.31.87.190:41935
--webserver_port=37581
--webserver_interface=127.31.87.131
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.131.keytab
--principal=oryx/127.31.87.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:40667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.31.87.190:41935,127.31.87.189:34383
--tserver_master_addrs=127.31.87.190:41935,127.31.87.189:34383,127.31.87.188:37451 with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:32:04.758625 11199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:32:04.758847 11199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:32:04.758888 11199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:32:04.762732 11199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:32:04.762778 11199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:32:04.762845 11199 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.131
I20260808 22:32:04.767848 11199 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:40667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.131.keytab
--principal=oryx/127.31.87.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.131:46167
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.87.131
--webserver_port=37581
--tserver_master_addrs=127.31.87.190:41935,127.31.87.189:34383,127.31.87.188:37451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11199
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.131
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:32:04.768787 11199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:32:04.769802 11199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:32:04.778883 11208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:04.778971 11207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:04.778961 11199 server_base.cc:1034] running on GCE node
W20260808 22:32:04.779098 11211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:04.779572 11199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:32:04.780148 11199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:32:04.781318 11199 hybrid_clock.cc:648] HybridClock initialized: now 1786228324781292 us; error 38 us; skew 500 ppm
Aug 08 22:32:04 dist-test-slave-266d krb5kdc[9358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228324, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.87.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:32:04.783902 11199 init.cc:377] Logged in from keytab as oryx/127.31.87.131@KRBTEST.COM (short username oryx)
I20260808 22:32:04.784749 11199 webserver.cc:460] Webserver started at http://127.31.87.131:37581/ using document root <none> and password file <none>
I20260808 22:32:04.785323 11199 fs_manager.cc:359] Metadata directory not provided
I20260808 22:32:04.785408 11199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:32:04.789003 11199 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:32:04.790895 11227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:04.792061 11199 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:32:04.792182 11199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-2/wal
uuid: "80ab594182ec4c6c95579360d4cc7210"
format_stamp: "Formatted at 2026-08-08 22:32:03 on dist-test-slave-266d"
server_key: "12aa4a892d5828f4c90f460c37f4f8b4"
server_key_iv: "de5d36bc1233b9767952b929be3ec814"
server_key_version: "encryptionkey@0"
I20260808 22:32:04.792680 11199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:32:04.805300 11199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:32:04.807740 11199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:32:04.807897 11199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:32:04.808405 11199 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:32:04.809242 11199 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:32:04.809291 11199 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:04.809341 11199 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:32:04.809370 11199 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:04.815616 10974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:04.811949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42128 (local address 127.31.87.188:37451)
0808 22:32:04.812099 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:04.812104 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:04.812123 (+ 19us) server_negotiation.cc:407] Connection header received
0808 22:32:04.812214 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:04.812218 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:04.812288 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:04.812405 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:32:04.812727 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.813517 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.814597 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.815079 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.815177 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:04.815326 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:04.815414 (+ 88us) server_negotiation.cc:299] Negotiation successful
0808 22:32:04.815482 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 22:32:04.815620 11247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:04.811638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.188:37451 (local address 127.0.0.1:42128)
0808 22:32:04.811984 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:04.811999 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:04.812107 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:04.812420 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:04.812423 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:04.812447 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:32:04.812558 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:04.812566 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.813637 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.813641 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:04.814473 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:04.814485 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.815200 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.815204 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:04.815306 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:04.815331 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 22:32:04.815403 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 22:32:04.815482 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":59,"threads_started":1}
I20260808 22:32:04.816185 10682 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260808 22:32:04.816550 11340 sys_catalog.cc:455] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ecff71ab1744a56b7959dc9072f4b64. Latest consensus state: current_term: 1 leader_uuid: "7ecff71ab1744a56b7959dc9072f4b64" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "396475d452f242d59b60b887d7c332f0" member_type: NON_VOTER last_known_addr { host: "127.31.87.188" port: 37451 } attrs { promote: true } } }
I20260808 22:32:04.816704 11340 sys_catalog.cc:458] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0 [sys.catalog]: This master's current role is: LEARNER
I20260808 22:32:04.821909 11199 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.131:46167
I20260808 22:32:04.821924 11398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.131:46167 every 8 connection(s)
I20260808 22:32:04.822842 11199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
I20260808 22:32:04.825503 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 11199
Aug 08 22:32:04 dist-test-slave-266d krb5kdc[9358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228324, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.87.131@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM
Aug 08 22:32:04 dist-test-slave-266d krb5kdc[9358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228324, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.87.131@KRBTEST.COM for oryx/127.31.87.188@KRBTEST.COM
I20260808 22:32:04.838682 11426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:04.828549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40170 (local address 127.31.87.189:34383)
0808 22:32:04.829119 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:04.829123 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:04.829488 (+ 365us) server_negotiation.cc:407] Connection header received
0808 22:32:04.829531 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:04.829534 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:04.829580 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:04.829664 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:04.830516 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.831313 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.832335 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.832660 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.832755 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:04.834228 (+ 1473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:04.834232 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:04.834234 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:04.834260 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:04.836333 (+ 2073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:04.836867 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:04.836870 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:04.836872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:04.836959 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:04.837318 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:04.837322 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:04.837324 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:04.837515 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:04.838199 (+ 684us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:04.838319 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 22:32:04.838451 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":83,"threads_started":1}
I20260808 22:32:04.839767 10974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:04.825240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:53183 (local address 127.31.87.188:37451)
0808 22:32:04.825358 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:04.825362 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:04.826407 (+ 1045us) server_negotiation.cc:407] Connection header received
0808 22:32:04.827114 (+ 707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:04.827117 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:04.827166 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:04.827238 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:04.830029 (+ 2791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.830778 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.831830 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.832127 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.832196 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:04.835651 (+ 3455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:04.835654 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:04.835657 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:04.835682 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:04.837528 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:04.838561 (+ 1033us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:04.838565 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:04.838567 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:04.838646 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:04.838936 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:04.838939 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:04.838941 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:04.839129 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:04.839208 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:04.839533 (+ 325us) server_negotiation.cc:299] Negotiation successful
0808 22:32:04.839630 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 22:32:04.840368 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:04.824892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:57665 (local address 127.31.87.190:41935)
0808 22:32:04.825123 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:04.825127 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:04.826463 (+ 1336us) server_negotiation.cc:407] Connection header received
0808 22:32:04.826854 (+ 391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:04.826857 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:04.826910 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:04.826998 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:04.828585 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.829400 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.831062 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.831406 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.831495 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:04.834992 (+ 3497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:04.834996 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:04.834998 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:04.835022 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:04.836814 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:04.837648 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:04.837652 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:04.837653 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:04.837736 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:04.838268 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:04.838271 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:04.838273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:04.838467 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:04.838548 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:04.840039 (+ 1491us) server_negotiation.cc:299] Negotiation successful
0808 22:32:04.840198 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136}
I20260808 22:32:04.840729 11408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:04.825116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41935 (local address 127.31.87.131:57665)
0808 22:32:04.826384 (+ 1268us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:04.826397 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:04.826641 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:04.827170 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:04.827178 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:04.827710 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:04.828433 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:04.828451 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.829560 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.829564 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:04.830948 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:04.830955 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.831644 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.831648 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:04.831829 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:04.832445 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:04.832464 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:04.834818 (+ 2354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:32:04.836940 (+ 2122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:04.836946 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:04.836960 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:04.837545 (+ 585us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:04.837963 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:04.837967 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:04.837969 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:04.838167 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:04.838598 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:32:04.838605 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:32:04.839104 (+ 499us) client_negotiation.cc:769] Sending connection context
0808 22:32:04.839274 (+ 170us) client_negotiation.cc:240] Negotiation successful
0808 22:32:04.839564 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1184,"mutex_wait_us":946,"thread_start_us":99,"threads_started":1}
I20260808 22:32:04.840729 11411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:04.825117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.188:37451 (local address 127.31.87.131:53183)
0808 22:32:04.826316 (+ 1199us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:04.826330 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:04.826733 (+ 403us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:04.827266 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:04.827269 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:04.827912 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:04.829905 (+ 1993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:04.829912 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.830911 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.830914 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:04.831691 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:04.831699 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.832345 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.832348 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:04.832455 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:04.832996 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:04.833011 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:04.835499 (+ 2488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:32:04.838213 (+ 2714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:04.838216 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:04.838218 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:04.838449 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:04.838748 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:04.838751 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:04.838753 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:04.838841 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:04.839274 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:32:04.839277 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:32:04.839358 (+ 81us) client_negotiation.cc:769] Sending connection context
0808 22:32:04.839938 (+ 580us) client_negotiation.cc:240] Negotiation successful
0808 22:32:04.840067 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1098,"mutex_wait_us":1778,"thread_start_us":222,"threads_started":1}
I20260808 22:32:04.842010 11400 heartbeater.cc:347] Connected to a master server at 127.31.87.188:37451
I20260808 22:32:04.842227 11400 heartbeater.cc:464] Registering TS with master...
I20260808 22:32:04.842742 11400 heartbeater.cc:511] Master 127.31.87.188:37451 requested a full tablet report, sending...
Aug 08 22:32:04 dist-test-slave-266d krb5kdc[9358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228324, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.87.131@KRBTEST.COM for oryx/127.31.87.189@KRBTEST.COM
I20260808 22:32:04.843266 11401 heartbeater.cc:347] Connected to a master server at 127.31.87.190:41935
I20260808 22:32:04.843336 11401 heartbeater.cc:464] Registering TS with master...
I20260808 22:32:04.843448 11423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:04.827883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42132 (local address 127.31.87.188:37451)
0808 22:32:04.828091 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:04.828095 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:04.828454 (+ 359us) server_negotiation.cc:407] Connection header received
0808 22:32:04.828640 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:04.828643 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:04.828688 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:04.828748 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:04.829765 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.830514 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.831635 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.831999 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.832068 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:04.835866 (+ 3798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:04.835869 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:04.835872 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:04.835898 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:04.838107 (+ 2209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:04.842134 (+ 4027us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:04.842138 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:04.842140 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:04.842224 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:04.842556 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:04.842559 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:04.842561 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:04.842745 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:04.842853 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:04.843175 (+ 322us) server_negotiation.cc:299] Negotiation successful
0808 22:32:04.843300 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":83,"threads_started":1}
I20260808 22:32:04.843478 11401 heartbeater.cc:511] Master 127.31.87.190:41935 requested a full tablet report, sending...
I20260808 22:32:04.844175 9423 ts_manager.cc:194] Re-registered known tserver with Master: 80ab594182ec4c6c95579360d4cc7210 (127.31.87.131:46167)
I20260808 22:32:04.844188 11422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:04.827575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46704 (local address 127.31.87.190:41935)
0808 22:32:04.833986 (+ 6411us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:04.833990 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:04.834002 (+ 12us) server_negotiation.cc:407] Connection header received
0808 22:32:04.834036 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:04.834039 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:04.834091 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:04.834835 (+ 744us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:04.834937 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.835705 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.837776 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.838447 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.839851 (+ 1404us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:04.840072 (+ 221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:04.840076 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:04.840079 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:04.840101 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:04.841892 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:04.842445 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:04.842448 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:04.842450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:04.842545 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:04.842920 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:04.842923 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:04.842925 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:04.843148 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:04.843786 (+ 638us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:04.843873 (+ 87us) server_negotiation.cc:299] Negotiation successful
0808 22:32:04.844001 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6341,"thread_start_us":71,"threads_started":1}
I20260808 22:32:04.844851 9423 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.87.131@KRBTEST.COM'} at 127.31.87.131:57665
I20260808 22:32:04.845278 10667 ts_manager.cc:194] Registered new tserver with Master: 80ab594182ec4c6c95579360d4cc7210 (127.31.87.131:46167)
I20260808 22:32:04.847371 11407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:04.825117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.189:34383 (local address 127.31.87.131:47055)
0808 22:32:04.825598 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:04.825646 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:04.826917 (+ 1271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:04.828142 (+ 1225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:04.828145 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:04.828303 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:04.828454 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:04.828460 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.829711 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.829714 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:04.834326 (+ 4612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:04.834338 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.835031 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.835035 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:04.835124 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:04.835997 (+ 873us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:04.836011 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:04.843316 (+ 7305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:32:04.845455 (+ 2139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:04.845458 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:04.845460 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:04.845701 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:04.846223 (+ 522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:04.846226 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:04.846228 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:04.846326 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:04.846906 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:32:04.846909 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:32:04.846990 (+ 81us) client_negotiation.cc:769] Sending connection context
0808 22:32:04.847068 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 22:32:04.847202 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"mutex_wait_us":3696,"thread_start_us":144,"threads_started":1}
I20260808 22:32:04.847383 10558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:04.824786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:47055 (local address 127.31.87.189:34383)
0808 22:32:04.825008 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:04.825012 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:04.825736 (+ 724us) server_negotiation.cc:407] Connection header received
0808 22:32:04.827027 (+ 1291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:04.827030 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:04.827083 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:04.827136 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:04.828599 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.829410 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.834509 (+ 5099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.834829 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.834903 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:04.843471 (+ 8568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:04.843475 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:04.843478 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:04.843503 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:04.845321 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:04.845829 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:04.845833 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:04.845834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:04.845926 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:04.846626 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:04.846629 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:04.846631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:04.846811 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:04.846900 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:04.847147 (+ 247us) server_negotiation.cc:299] Negotiation successful
0808 22:32:04.847243 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146}
I20260808 22:32:04.848052 11404 heartbeater.cc:347] Connected to a master server at 127.31.87.189:34383
I20260808 22:32:04.848127 11404 heartbeater.cc:464] Registering TS with master...
I20260808 22:32:04.848281 11404 heartbeater.cc:511] Master 127.31.87.189:34383 requested a full tablet report, sending...
I20260808 22:32:04.848846 10360 ts_manager.cc:194] Registered new tserver with Master: 80ab594182ec4c6c95579360d4cc7210 (127.31.87.131:46167)
I20260808 22:32:04.847733 9430 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46704:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:32:04.850924 9430 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:32:04.853425 11217 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7ecff71ab1744a56b7959dc9072f4b64" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "396475d452f242d59b60b887d7c332f0" member_type: NON_VOTER last_known_addr { host: "127.31.87.188" port: 37451 } attrs { promote: true } } }
I20260808 22:32:04.853549 11217 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 [sys.catalog]: This master's current role is: LEADER
I20260808 22:32:04.855121 11217 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64: attempting to promote NON_VOTER 396475d452f242d59b60b887d7c332f0 to VOTER
I20260808 22:32:04.855430 11217 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "396475d452f242d59b60b887d7c332f0" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37451 } attrs { promote: false } }
I20260808 22:32:04.856485 10682 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0 [term 1 LEARNER]: Refusing update from remote peer 7ecff71ab1744a56b7959dc9072f4b64: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260808 22:32:04.856511 10374 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21 [term 1 FOLLOWER]: Refusing update from remote peer 7ecff71ab1744a56b7959dc9072f4b64: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260808 22:32:04.856798 11431 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "396475d452f242d59b60b887d7c332f0" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37451 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260808 22:32:04.856917 11430 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260808 22:32:04.858532 11430 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 396475d452f242d59b60b887d7c332f0 (127.31.87.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "396475d452f242d59b60b887d7c332f0" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37451 } attrs { promote: false } } }
I20260808 22:32:04.859030 10374 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 396475d452f242d59b60b887d7c332f0 (127.31.87.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "396475d452f242d59b60b887d7c332f0" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37451 } attrs { promote: false } } }
I20260808 22:32:04.859200 10682 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 396475d452f242d59b60b887d7c332f0 (127.31.87.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "396475d452f242d59b60b887d7c332f0" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37451 } attrs { promote: false } } }
I20260808 22:32:04.859932 11430 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "7ecff71ab1744a56b7959dc9072f4b64" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "396475d452f242d59b60b887d7c332f0" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37451 } attrs { promote: false } } }
I20260808 22:32:04.860023 11430 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 [sys.catalog]: This master's current role is: LEADER
I20260808 22:32:04.860167 11430 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7ecff71ab1744a56b7959dc9072f4b64" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "396475d452f242d59b60b887d7c332f0" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37451 } attrs { promote: false } } }
I20260808 22:32:04.860253 11430 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 [sys.catalog]: This master's current role is: LEADER
I20260808 22:32:04.860343 11430 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7ecff71ab1744a56b7959dc9072f4b64" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "396475d452f242d59b60b887d7c332f0" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37451 } attrs { promote: false } } }
I20260808 22:32:04.860435 11430 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64 [sys.catalog]: This master's current role is: LEADER
I20260808 22:32:04.860808 11340 sys_catalog.cc:455] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7ecff71ab1744a56b7959dc9072f4b64" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "396475d452f242d59b60b887d7c332f0" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37451 } attrs { promote: false } } }
I20260808 22:32:04.860922 11340 sys_catalog.cc:458] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:32:04.862048 11441 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7ecff71ab1744a56b7959dc9072f4b64" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "396475d452f242d59b60b887d7c332f0" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37451 } attrs { promote: false } } }
I20260808 22:32:04.862156 11441 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:32:04.872171 11447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:04.866524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56910 (local address 127.31.87.129:44881)
0808 22:32:04.866987 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:04.867009 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:04.867040 (+ 31us) server_negotiation.cc:407] Connection header received
0808 22:32:04.867142 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:04.867148 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:04.867343 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:04.867510 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:32:04.867894 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.868998 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.870432 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.871281 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.871396 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:04.871547 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:04.871891 (+ 344us) server_negotiation.cc:299] Negotiation successful
0808 22:32:04.871995 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":128,"threads_started":1}
I20260808 22:32:04.872190 11247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:04.866089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.131:46167 (local address 127.0.0.1:54178)
0808 22:32:04.866215 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:04.866229 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:04.866309 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:04.867555 (+ 1246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:04.867558 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:04.867572 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:32:04.867659 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:04.867666 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.868969 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.868973 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:04.870115 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:04.870127 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.871818 (+ 1691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.871822 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:04.871904 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:04.871924 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 22:32:04.871997 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 22:32:04.872067 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":16}
I20260808 22:32:04.872447 11444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:04.866649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.130:35045 (local address 127.0.0.1:54810)
0808 22:32:04.867179 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:04.867190 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:04.867259 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:04.868304 (+ 1045us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:04.868308 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:04.868328 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:32:04.868474 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:04.868482 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.869733 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.869737 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:04.870848 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:04.870859 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.872034 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.872039 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:04.872127 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:04.872154 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 22:32:04.872235 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 22:32:04.872311 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":75,"threads_started":1}
I20260808 22:32:04.872603 11449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:04.866344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:44881 (local address 127.0.0.1:56910)
0808 22:32:04.866910 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:04.866923 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:04.867013 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:04.867499 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:04.867502 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:04.867519 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:32:04.867775 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:04.867782 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.869147 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.869150 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:04.870317 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:04.870328 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.871624 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.871628 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:04.871718 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:04.871748 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 22:32:04.872385 (+ 637us) client_negotiation.cc:240] Negotiation successful
0808 22:32:04.872452 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"mutex_wait_us":4,"thread_start_us":101,"threads_started":1}
I20260808 22:32:04.872866 11448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:04.866597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54178 (local address 127.31.87.131:46167)
0808 22:32:04.867119 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:04.867131 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:04.867151 (+ 20us) server_negotiation.cc:407] Connection header received
0808 22:32:04.867218 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:04.867223 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:04.867424 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:04.867637 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:32:04.867744 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.868868 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.871024 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.871704 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.871807 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:04.871934 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:04.872024 (+ 90us) server_negotiation.cc:299] Negotiation successful
0808 22:32:04.872099 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":89,"threads_started":1}
I20260808 22:32:04.872961 11450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:04.866842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54810 (local address 127.31.87.130:35045)
0808 22:32:04.867834 (+ 992us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:04.867850 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:04.867874 (+ 24us) server_negotiation.cc:407] Connection header received
0808 22:32:04.867950 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:04.867955 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:04.868158 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:04.868316 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:32:04.868590 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.869619 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.870958 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:04.871720 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:04.871832 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:04.871969 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:04.872704 (+ 735us) server_negotiation.cc:299] Negotiation successful
0808 22:32:04.872810 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":885,"thread_start_us":181,"threads_started":1}
I20260808 22:32:04.875216 10855 tablet_service.cc:1467] Processing CreateTablet for tablet 7d251422aeba4feb896c4438f30f6a4d (DEFAULT_TABLE table=test-table [id=657bf01249ba43edb53a7dd7d9076996]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:32:04.875393 11073 tablet_service.cc:1467] Processing CreateTablet for tablet 7d251422aeba4feb896c4438f30f6a4d (DEFAULT_TABLE table=test-table [id=657bf01249ba43edb53a7dd7d9076996]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:32:04.876139 10855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d251422aeba4feb896c4438f30f6a4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:32:04.876297 11073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d251422aeba4feb896c4438f30f6a4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:32:04.879033 11299 tablet_service.cc:1467] Processing CreateTablet for tablet 7d251422aeba4feb896c4438f30f6a4d (DEFAULT_TABLE table=test-table [id=657bf01249ba43edb53a7dd7d9076996]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:32:04.880057 11299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d251422aeba4feb896c4438f30f6a4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:32:04.882468 11451 tablet_bootstrap.cc:492] T 7d251422aeba4feb896c4438f30f6a4d P 2d486908e4e54ccbb78867ae276bf39f: Bootstrap starting.
I20260808 22:32:04.884605 11451 tablet_bootstrap.cc:654] T 7d251422aeba4feb896c4438f30f6a4d P 2d486908e4e54ccbb78867ae276bf39f: Neither blocks nor log segments found. Creating new log.
I20260808 22:32:04.884675 11453 tablet_bootstrap.cc:492] T 7d251422aeba4feb896c4438f30f6a4d P 0490e64eb57d43268a8e1c61d601e835: Bootstrap starting.
I20260808 22:32:04.885599 11451 log.cc:824] T 7d251422aeba4feb896c4438f30f6a4d P 2d486908e4e54ccbb78867ae276bf39f: Log is configured to *not* fsync() on all Append() calls
I20260808 22:32:04.886801 11453 tablet_bootstrap.cc:654] T 7d251422aeba4feb896c4438f30f6a4d P 0490e64eb57d43268a8e1c61d601e835: Neither blocks nor log segments found. Creating new log.
I20260808 22:32:04.887529 11455 tablet_bootstrap.cc:492] T 7d251422aeba4feb896c4438f30f6a4d P 80ab594182ec4c6c95579360d4cc7210: Bootstrap starting.
I20260808 22:32:04.887595 11453 log.cc:824] T 7d251422aeba4feb896c4438f30f6a4d P 0490e64eb57d43268a8e1c61d601e835: Log is configured to *not* fsync() on all Append() calls
I20260808 22:32:04.887725 11451 tablet_bootstrap.cc:492] T 7d251422aeba4feb896c4438f30f6a4d P 2d486908e4e54ccbb78867ae276bf39f: No bootstrap required, opened a new log
I20260808 22:32:04.887935 11451 ts_tablet_manager.cc:1397] T 7d251422aeba4feb896c4438f30f6a4d P 2d486908e4e54ccbb78867ae276bf39f: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260808 22:32:04.889621 11453 tablet_bootstrap.cc:492] T 7d251422aeba4feb896c4438f30f6a4d P 0490e64eb57d43268a8e1c61d601e835: No bootstrap required, opened a new log
I20260808 22:32:04.889647 11455 tablet_bootstrap.cc:654] T 7d251422aeba4feb896c4438f30f6a4d P 80ab594182ec4c6c95579360d4cc7210: Neither blocks nor log segments found. Creating new log.
I20260808 22:32:04.889819 11453 ts_tablet_manager.cc:1397] T 7d251422aeba4feb896c4438f30f6a4d P 0490e64eb57d43268a8e1c61d601e835: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 22:32:04.890292 11455 log.cc:824] T 7d251422aeba4feb896c4438f30f6a4d P 80ab594182ec4c6c95579360d4cc7210: Log is configured to *not* fsync() on all Append() calls
I20260808 22:32:04.892314 11455 tablet_bootstrap.cc:492] T 7d251422aeba4feb896c4438f30f6a4d P 80ab594182ec4c6c95579360d4cc7210: No bootstrap required, opened a new log
I20260808 22:32:04.892493 11455 ts_tablet_manager.cc:1397] T 7d251422aeba4feb896c4438f30f6a4d P 80ab594182ec4c6c95579360d4cc7210: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 22:32:04.892618 11451 raft_consensus.cc:348] T 7d251422aeba4feb896c4438f30f6a4d P 2d486908e4e54ccbb78867ae276bf39f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80ab594182ec4c6c95579360d4cc7210" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 46167 } } peers { permanent_uuid: "2d486908e4e54ccbb78867ae276bf39f" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 44881 } } peers { permanent_uuid: "0490e64eb57d43268a8e1c61d601e835" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 35045 } }
I20260808 22:32:04.892891 11451 raft_consensus.cc:374] T 7d251422aeba4feb896c4438f30f6a4d P 2d486908e4e54ccbb78867ae276bf39f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:32:04.892954 11451 raft_consensus.cc:729] T 7d251422aeba4feb896c4438f30f6a4d P 2d486908e4e54ccbb78867ae276bf39f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d486908e4e54ccbb78867ae276bf39f, State: Initialized, Role: FOLLOWER
I20260808 22:32:04.893457 11451 consensus_queue.cc:260] T 7d251422aeba4feb896c4438f30f6a4d P 2d486908e4e54ccbb78867ae276bf39f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80ab594182ec4c6c95579360d4cc7210" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 46167 } } peers { permanent_uuid: "2d486908e4e54ccbb78867ae276bf39f" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 44881 } } peers { permanent_uuid: "0490e64eb57d43268a8e1c61d601e835" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 35045 } }
I20260808 22:32:04.894526 11451 ts_tablet_manager.cc:1428] T 7d251422aeba4feb896c4438f30f6a4d P 2d486908e4e54ccbb78867ae276bf39f: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260808 22:32:04.894333 11453 raft_consensus.cc:348] T 7d251422aeba4feb896c4438f30f6a4d P 0490e64eb57d43268a8e1c61d601e835 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80ab594182ec4c6c95579360d4cc7210" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 46167 } } peers { permanent_uuid: "2d486908e4e54ccbb78867ae276bf39f" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 44881 } } peers { permanent_uuid: "0490e64eb57d43268a8e1c61d601e835" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 35045 } }
I20260808 22:32:04.894603 11453 raft_consensus.cc:374] T 7d251422aeba4feb896c4438f30f6a4d P 0490e64eb57d43268a8e1c61d601e835 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:32:04.894654 11453 raft_consensus.cc:729] T 7d251422aeba4feb896c4438f30f6a4d P 0490e64eb57d43268a8e1c61d601e835 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0490e64eb57d43268a8e1c61d601e835, State: Initialized, Role: FOLLOWER
I20260808 22:32:04.895138 10966 heartbeater.cc:503] Master 127.31.87.190:41935 was elected leader, sending a full tablet report...
I20260808 22:32:04.895565 11453 consensus_queue.cc:260] T 7d251422aeba4feb896c4438f30f6a4d P 0490e64eb57d43268a8e1c61d601e835 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80ab594182ec4c6c95579360d4cc7210" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 46167 } } peers { permanent_uuid: "2d486908e4e54ccbb78867ae276bf39f" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 44881 } } peers { permanent_uuid: "0490e64eb57d43268a8e1c61d601e835" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 35045 } }
I20260808 22:32:04.896294 11183 heartbeater.cc:503] Master 127.31.87.190:41935 was elected leader, sending a full tablet report...
I20260808 22:32:04.896787 11453 ts_tablet_manager.cc:1428] T 7d251422aeba4feb896c4438f30f6a4d P 0490e64eb57d43268a8e1c61d601e835: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260808 22:32:04.907932 11455 raft_consensus.cc:348] T 7d251422aeba4feb896c4438f30f6a4d P 80ab594182ec4c6c95579360d4cc7210 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80ab594182ec4c6c95579360d4cc7210" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 46167 } } peers { permanent_uuid: "2d486908e4e54ccbb78867ae276bf39f" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 44881 } } peers { permanent_uuid: "0490e64eb57d43268a8e1c61d601e835" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 35045 } }
I20260808 22:32:04.908244 11455 raft_consensus.cc:374] T 7d251422aeba4feb896c4438f30f6a4d P 80ab594182ec4c6c95579360d4cc7210 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:32:04.908322 11455 raft_consensus.cc:729] T 7d251422aeba4feb896c4438f30f6a4d P 80ab594182ec4c6c95579360d4cc7210 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80ab594182ec4c6c95579360d4cc7210, State: Initialized, Role: FOLLOWER
I20260808 22:32:04.908695 11455 consensus_queue.cc:260] T 7d251422aeba4feb896c4438f30f6a4d P 80ab594182ec4c6c95579360d4cc7210 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80ab594182ec4c6c95579360d4cc7210" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 46167 } } peers { permanent_uuid: "2d486908e4e54ccbb78867ae276bf39f" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 44881 } } peers { permanent_uuid: "0490e64eb57d43268a8e1c61d601e835" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 35045 } }
I20260808 22:32:04.909302 11401 heartbeater.cc:503] Master 127.31.87.190:41935 was elected leader, sending a full tablet report...
I20260808 22:32:04.909634 11455 ts_tablet_manager.cc:1428] T 7d251422aeba4feb896c4438f30f6a4d P 80ab594182ec4c6c95579360d4cc7210: Time spent starting tablet: real 0.017s user 0.005s sys 0.000s
W20260808 22:32:04.937686 10968 tablet.cc:2369] T 7d251422aeba4feb896c4438f30f6a4d P 2d486908e4e54ccbb78867ae276bf39f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 22:32:05.074848 11406 tablet.cc:2369] T 7d251422aeba4feb896c4438f30f6a4d P 80ab594182ec4c6c95579360d4cc7210: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:32:05.111229 11464 raft_consensus.cc:482] T 7d251422aeba4feb896c4438f30f6a4d P 0490e64eb57d43268a8e1c61d601e835 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:32:05.111438 11464 raft_consensus.cc:504] T 7d251422aeba4feb896c4438f30f6a4d P 0490e64eb57d43268a8e1c61d601e835 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80ab594182ec4c6c95579360d4cc7210" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 46167 } } peers { permanent_uuid: "2d486908e4e54ccbb78867ae276bf39f" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 44881 } } peers { permanent_uuid: "0490e64eb57d43268a8e1c61d601e835" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 35045 } }
I20260808 22:32:05.112504 11464 leader_election.cc:288] T 7d251422aeba4feb896c4438f30f6a4d P 0490e64eb57d43268a8e1c61d601e835 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80ab594182ec4c6c95579360d4cc7210 (127.31.87.131:46167), 2d486908e4e54ccbb78867ae276bf39f (127.31.87.129:44881)
W20260808 22:32:05.114723 11188 tablet.cc:2369] T 7d251422aeba4feb896c4438f30f6a4d P 0490e64eb57d43268a8e1c61d601e835: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:32:05.117391 11447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:05.113166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:49357 (local address 127.31.87.129:44881)
0808 22:32:05.113292 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:05.113295 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:05.113308 (+ 13us) server_negotiation.cc:407] Connection header received
0808 22:32:05.113356 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:05.113358 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:05.113395 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:05.113457 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:32:05.113687 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:05.114408 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:05.116073 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:05.116687 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:05.116807 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:05.116858 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:05.117192 (+ 334us) server_negotiation.cc:299] Negotiation successful
0808 22:32:05.117271 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 22:32:05.117390 11448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:05.112851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:45539 (local address 127.31.87.131:46167)
0808 22:32:05.112984 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:05.112989 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:05.113004 (+ 15us) server_negotiation.cc:407] Connection header received
0808 22:32:05.113123 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:05.113127 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:05.113175 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:05.113248 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:32:05.113586 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:05.114378 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:05.116073 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:05.116687 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:05.116807 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:05.116858 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:05.117192 (+ 334us) server_negotiation.cc:299] Negotiation successful
0808 22:32:05.117261 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 22:32:05.117478 11189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:05.113030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:44881 (local address 127.31.87.130:49357)
0808 22:32:05.113185 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:05.113196 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:05.113254 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:05.113488 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:05.113491 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:05.113505 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:32:05.113593 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:05.113599 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:05.114521 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:05.114526 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:05.115930 (+ 1404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:05.115943 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:05.116807 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:05.116811 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:05.117028 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:05.117107 (+ 79us) client_negotiation.cc:769] Sending connection context
0808 22:32:05.117178 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 22:32:05.117300 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"mutex_wait_us":178}
I20260808 22:32:05.117501 11191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:05.112733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.131:46167 (local address 127.31.87.130:45539)
0808 22:32:05.112875 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:05.112890 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:05.112992 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:05.113344 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:05.113346 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:05.113361 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:32:05.113474 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:05.113480 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:05.114521 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:05.114526 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:05.115935 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:05.115943 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:05.116807 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:05.116811 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:05.117022 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:05.117068 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 22:32:05.117178 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 22:32:05.117294 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":150}
I20260808 22:32:05.118151 10882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d251422aeba4feb896c4438f30f6a4d" candidate_uuid: "0490e64eb57d43268a8e1c61d601e835" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d486908e4e54ccbb78867ae276bf39f" is_pre_election: true
I20260808 22:32:05.118163 11333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d251422aeba4feb896c4438f30f6a4d" candidate_uuid: "0490e64eb57d43268a8e1c61d601e835" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80ab594182ec4c6c95579360d4cc7210" is_pre_election: true
I20260808 22:32:05.118413 10882 raft_consensus.cc:2454] T 7d251422aeba4feb896c4438f30f6a4d P 2d486908e4e54ccbb78867ae276bf39f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0490e64eb57d43268a8e1c61d601e835 in term 0.
I20260808 22:32:05.118427 11333 raft_consensus.cc:2454] T 7d251422aeba4feb896c4438f30f6a4d P 80ab594182ec4c6c95579360d4cc7210 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0490e64eb57d43268a8e1c61d601e835 in term 0.
I20260808 22:32:05.118844 11002 leader_election.cc:302] T 7d251422aeba4feb896c4438f30f6a4d P 0490e64eb57d43268a8e1c61d601e835 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0490e64eb57d43268a8e1c61d601e835, 2d486908e4e54ccbb78867ae276bf39f; no voters:
I20260808 22:32:05.119112 11464 raft_consensus.cc:2781] T 7d251422aeba4feb896c4438f30f6a4d P 0490e64eb57d43268a8e1c61d601e835 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:32:05.119187 11464 raft_consensus.cc:482] T 7d251422aeba4feb896c4438f30f6a4d P 0490e64eb57d43268a8e1c61d601e835 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:32:05.119233 11464 raft_consensus.cc:3037] T 7d251422aeba4feb896c4438f30f6a4d P 0490e64eb57d43268a8e1c61d601e835 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:32:05.120244 11464 raft_consensus.cc:504] T 7d251422aeba4feb896c4438f30f6a4d P 0490e64eb57d43268a8e1c61d601e835 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80ab594182ec4c6c95579360d4cc7210" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 46167 } } peers { permanent_uuid: "2d486908e4e54ccbb78867ae276bf39f" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 44881 } } peers { permanent_uuid: "0490e64eb57d43268a8e1c61d601e835" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 35045 } }
I20260808 22:32:05.120540 11464 leader_election.cc:288] T 7d251422aeba4feb896c4438f30f6a4d P 0490e64eb57d43268a8e1c61d601e835 [CANDIDATE]: Term 1 election: Requested vote from peers 80ab594182ec4c6c95579360d4cc7210 (127.31.87.131:46167), 2d486908e4e54ccbb78867ae276bf39f (127.31.87.129:44881)
I20260808 22:32:05.120889 11333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d251422aeba4feb896c4438f30f6a4d" candidate_uuid: "0490e64eb57d43268a8e1c61d601e835" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80ab594182ec4c6c95579360d4cc7210"
I20260808 22:32:05.120929 10882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d251422aeba4feb896c4438f30f6a4d" candidate_uuid: "0490e64eb57d43268a8e1c61d601e835" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d486908e4e54ccbb78867ae276bf39f"
I20260808 22:32:05.121001 11333 raft_consensus.cc:3037] T 7d251422aeba4feb896c4438f30f6a4d P 80ab594182ec4c6c95579360d4cc7210 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:32:05.121031 10882 raft_consensus.cc:3037] T 7d251422aeba4feb896c4438f30f6a4d P 2d486908e4e54ccbb78867ae276bf39f [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:32:05.121975 10882 raft_consensus.cc:2454] T 7d251422aeba4feb896c4438f30f6a4d P 2d486908e4e54ccbb78867ae276bf39f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0490e64eb57d43268a8e1c61d601e835 in term 1.
I20260808 22:32:05.122133 11333 raft_consensus.cc:2454] T 7d251422aeba4feb896c4438f30f6a4d P 80ab594182ec4c6c95579360d4cc7210 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0490e64eb57d43268a8e1c61d601e835 in term 1.
I20260808 22:32:05.122222 11002 leader_election.cc:302] T 7d251422aeba4feb896c4438f30f6a4d P 0490e64eb57d43268a8e1c61d601e835 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0490e64eb57d43268a8e1c61d601e835, 2d486908e4e54ccbb78867ae276bf39f; no voters:
I20260808 22:32:05.122372 11464 raft_consensus.cc:2781] T 7d251422aeba4feb896c4438f30f6a4d P 0490e64eb57d43268a8e1c61d601e835 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:32:05.122545 11464 raft_consensus.cc:686] T 7d251422aeba4feb896c4438f30f6a4d P 0490e64eb57d43268a8e1c61d601e835 [term 1 LEADER]: Becoming Leader. State: Replica: 0490e64eb57d43268a8e1c61d601e835, State: Running, Role: LEADER
I20260808 22:32:05.122853 11464 consensus_queue.cc:237] T 7d251422aeba4feb896c4438f30f6a4d P 0490e64eb57d43268a8e1c61d601e835 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80ab594182ec4c6c95579360d4cc7210" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 46167 } } peers { permanent_uuid: "2d486908e4e54ccbb78867ae276bf39f" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 44881 } } peers { permanent_uuid: "0490e64eb57d43268a8e1c61d601e835" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 35045 } }
I20260808 22:32:05.126101 9423 catalog_manager.cc:5495] T 7d251422aeba4feb896c4438f30f6a4d P 0490e64eb57d43268a8e1c61d601e835 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0490e64eb57d43268a8e1c61d601e835 (127.31.87.130). New cstate: current_term: 1 leader_uuid: "0490e64eb57d43268a8e1c61d601e835" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80ab594182ec4c6c95579360d4cc7210" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 46167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d486908e4e54ccbb78867ae276bf39f" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 44881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0490e64eb57d43268a8e1c61d601e835" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 35045 } health_report { overall_health: HEALTHY } } }
I20260808 22:32:05.165200 11450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:05.162191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54824 (local address 127.31.87.130:35045)
0808 22:32:05.162306 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:05.162309 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:05.162323 (+ 14us) server_negotiation.cc:407] Connection header received
0808 22:32:05.162397 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:05.162399 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:05.162453 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:05.162536 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:32:05.162717 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:05.163202 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:05.163978 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:05.164200 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:05.164278 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:05.164487 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:32:05.164891 (+ 404us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:32:05.164976 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:05.165003 (+ 27us) server_negotiation.cc:299] Negotiation successful
0808 22:32:05.165051 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 22:32:05.170862 11333 raft_consensus.cc:1261] T 7d251422aeba4feb896c4438f30f6a4d P 80ab594182ec4c6c95579360d4cc7210 [term 1 FOLLOWER]: Refusing update from remote peer 0490e64eb57d43268a8e1c61d601e835: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:32:05.170885 10882 raft_consensus.cc:1261] T 7d251422aeba4feb896c4438f30f6a4d P 2d486908e4e54ccbb78867ae276bf39f [term 1 FOLLOWER]: Refusing update from remote peer 0490e64eb57d43268a8e1c61d601e835: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:32:05.171512 11464 consensus_queue.cc:1035] T 7d251422aeba4feb896c4438f30f6a4d P 0490e64eb57d43268a8e1c61d601e835 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d486908e4e54ccbb78867ae276bf39f" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 44881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:32:05.171744 11466 consensus_queue.cc:1035] T 7d251422aeba4feb896c4438f30f6a4d P 0490e64eb57d43268a8e1c61d601e835 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80ab594182ec4c6c95579360d4cc7210" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 46167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:32:05.179898 11469 mvcc.cc:205] Tried to move back new op lower bound from 7316391219892686848 to 7316391219705745408. Current Snapshot: MvccSnapshot[applied={T|T < 7316391219892686848}]
I20260808 22:32:05.181998 11470 mvcc.cc:205] Tried to move back new op lower bound from 7316391219892686848 to 7316391219705745408. Current Snapshot: MvccSnapshot[applied={T|T < 7316391219892686848}]
I20260808 22:32:05.183022 11472 mvcc.cc:205] Tried to move back new op lower bound from 7316391219892686848 to 7316391219705745408. Current Snapshot: MvccSnapshot[applied={T|T < 7316391219892686848 or (T in {7316391219892686848})}]
I20260808 22:32:05.189939 9423 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46704:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:32:05.190074 9423 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46704:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:32:05.193912 9423 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7ecff71ab1744a56b7959dc9072f4b64: Sending DeleteTablet for 3 replicas of tablet 7d251422aeba4feb896c4438f30f6a4d
I20260808 22:32:05.194664 11299 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d251422aeba4feb896c4438f30f6a4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:32:05 UTC) from {username='oryx', principal='oryx/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:54178
I20260808 22:32:05.194903 10855 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d251422aeba4feb896c4438f30f6a4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:32:05 UTC) from {username='oryx', principal='oryx/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:56910
I20260808 22:32:05.195227 11073 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d251422aeba4feb896c4438f30f6a4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:32:05 UTC) from {username='oryx', principal='oryx/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:54810
I20260808 22:32:05.195768 11483 tablet_replica.cc:330] T 7d251422aeba4feb896c4438f30f6a4d P 80ab594182ec4c6c95579360d4cc7210: stopping tablet replica
I20260808 22:32:05.195770 11485 tablet_replica.cc:330] T 7d251422aeba4feb896c4438f30f6a4d P 0490e64eb57d43268a8e1c61d601e835: stopping tablet replica
I20260808 22:32:05.196134 11483 raft_consensus.cc:2229] T 7d251422aeba4feb896c4438f30f6a4d P 80ab594182ec4c6c95579360d4cc7210 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:32:05.196134 11485 raft_consensus.cc:2229] T 7d251422aeba4feb896c4438f30f6a4d P 0490e64eb57d43268a8e1c61d601e835 [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:32:05.196615 11485 raft_consensus.cc:2258] T 7d251422aeba4feb896c4438f30f6a4d P 0490e64eb57d43268a8e1c61d601e835 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:32:05.197136 11484 tablet_replica.cc:330] T 7d251422aeba4feb896c4438f30f6a4d P 2d486908e4e54ccbb78867ae276bf39f: stopping tablet replica
I20260808 22:32:05.197497 11484 raft_consensus.cc:2229] T 7d251422aeba4feb896c4438f30f6a4d P 2d486908e4e54ccbb78867ae276bf39f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:32:05.197747 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 9380
I20260808 22:32:05.199411 11485 ts_tablet_manager.cc:1905] T 7d251422aeba4feb896c4438f30f6a4d P 0490e64eb57d43268a8e1c61d601e835: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:32:05.203058 11485 ts_tablet_manager.cc:1918] T 7d251422aeba4feb896c4438f30f6a4d P 0490e64eb57d43268a8e1c61d601e835: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:32:05.203219 11485 log.cc:1196] T 7d251422aeba4feb896c4438f30f6a4d P 0490e64eb57d43268a8e1c61d601e835: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-1/wal/wals/7d251422aeba4feb896c4438f30f6a4d
I20260808 22:32:05.203644 11485 ts_tablet_manager.cc:1939] T 7d251422aeba4feb896c4438f30f6a4d P 0490e64eb57d43268a8e1c61d601e835: Deleting consensus metadata
I20260808 22:32:05.205667 11484 raft_consensus.cc:2258] T 7d251422aeba4feb896c4438f30f6a4d P 2d486908e4e54ccbb78867ae276bf39f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:32:05.207854 11484 ts_tablet_manager.cc:1905] T 7d251422aeba4feb896c4438f30f6a4d P 2d486908e4e54ccbb78867ae276bf39f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:32:05.210913 10558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:05.197451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40184 (local address 127.31.87.189:34383)
0808 22:32:05.197580 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:05.197584 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:05.198424 (+ 840us) server_negotiation.cc:407] Connection header received
0808 22:32:05.198631 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:05.198634 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:05.198688 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:05.198741 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:05.199915 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:05.200675 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:05.201950 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:05.202313 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:05.202393 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:05.205209 (+ 2816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:05.205214 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:05.205217 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:05.205253 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:05.208315 (+ 3062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:05.209496 (+ 1181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:05.209500 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:05.209502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:05.209595 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:05.209894 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:05.209897 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:05.209899 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:05.210131 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:05.210249 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:05.210611 (+ 362us) server_negotiation.cc:299] Negotiation successful
0808 22:32:05.210742 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 22:32:05.211198 11484 ts_tablet_manager.cc:1918] T 7d251422aeba4feb896c4438f30f6a4d P 2d486908e4e54ccbb78867ae276bf39f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:32:05.211335 11484 log.cc:1196] T 7d251422aeba4feb896c4438f30f6a4d P 2d486908e4e54ccbb78867ae276bf39f: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-0/wal/wals/7d251422aeba4feb896c4438f30f6a4d
I20260808 22:32:05.211750 11484 ts_tablet_manager.cc:1939] T 7d251422aeba4feb896c4438f30f6a4d P 2d486908e4e54ccbb78867ae276bf39f: Deleting consensus metadata
I20260808 22:32:05.221844 11423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:05.197847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42138 (local address 127.31.87.188:37451)
0808 22:32:05.197974 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:05.197977 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:05.212807 (+ 14830us) server_negotiation.cc:407] Connection header received
0808 22:32:05.213005 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:05.213009 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:05.213071 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:05.213145 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:05.214057 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:05.214948 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:05.216115 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:05.216547 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:05.216661 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:05.218273 (+ 1612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:05.218277 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:05.218280 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:05.218310 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:05.220038 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:05.220506 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:05.220510 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:05.220512 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:05.220609 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:05.220878 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:05.220881 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:05.220883 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:05.221170 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:05.221294 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:05.221536 (+ 242us) server_negotiation.cc:299] Negotiation successful
0808 22:32:05.221676 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260808 22:32:05.224166 10786 connection.cc:535] server connection from 127.0.0.1:56910 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 22:32:05.224167 11003 connection.cc:535] server connection from 127.0.0.1:54810 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 22:32:05.224529 11483 raft_consensus.cc:2258] T 7d251422aeba4feb896c4438f30f6a4d P 80ab594182ec4c6c95579360d4cc7210 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:32:05.227123 11483 ts_tablet_manager.cc:1905] T 7d251422aeba4feb896c4438f30f6a4d P 80ab594182ec4c6c95579360d4cc7210: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:32:05.230095 11483 ts_tablet_manager.cc:1918] T 7d251422aeba4feb896c4438f30f6a4d P 80ab594182ec4c6c95579360d4cc7210: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:32:05.230283 11483 log.cc:1196] T 7d251422aeba4feb896c4438f30f6a4d P 80ab594182ec4c6c95579360d4cc7210: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-2/wal/wals/7d251422aeba4feb896c4438f30f6a4d
I20260808 22:32:05.230659 11483 ts_tablet_manager.cc:1939] T 7d251422aeba4feb896c4438f30f6a4d P 80ab594182ec4c6c95579360d4cc7210: Deleting consensus metadata
W20260808 22:32:05.231904 11230 connection.cc:441] server connection from 127.0.0.1:54178 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54178 (request call id 1) could send its response
I20260808 22:32:05.237968 11423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:05.228016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42154 (local address 127.31.87.188:37451)
0808 22:32:05.228119 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:05.228123 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:05.228360 (+ 237us) server_negotiation.cc:407] Connection header received
0808 22:32:05.228555 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:05.228558 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:05.228607 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:05.228659 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:05.229724 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:05.230544 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:05.231908 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:05.232253 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:05.232319 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:05.234416 (+ 2097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:05.234420 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:05.234422 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:05.234445 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:05.236201 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:05.236653 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:05.236657 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:05.236659 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:05.236741 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:05.237043 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:05.237048 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:05.237049 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:05.237323 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:05.237451 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:05.237688 (+ 237us) server_negotiation.cc:299] Negotiation successful
0808 22:32:05.237804 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 22:32:05.238847 10558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:05.227819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40190 (local address 127.31.87.189:34383)
0808 22:32:05.228122 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:05.228126 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:05.228674 (+ 548us) server_negotiation.cc:407] Connection header received
0808 22:32:05.228853 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:05.228856 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:05.228908 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:05.228960 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:05.229812 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:05.230564 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:05.231908 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:05.232244 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:05.232319 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:05.234381 (+ 2062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:05.234385 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:05.234388 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:05.234417 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:05.236109 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:05.236847 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:05.236850 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:05.236852 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:05.236930 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:05.237700 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:05.237704 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:05.237706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:05.237909 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:05.238000 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:05.238581 (+ 581us) server_negotiation.cc:299] Negotiation successful
0808 22:32:05.238699 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 22:32:06.188277 11504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:06.186474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41935 (local address 127.31.87.130:43517)
0808 22:32:06.187632 (+ 1158us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:06.188051 (+ 419us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":971,"thread_start_us":121,"threads_started":1}
I20260808 22:32:06.188446 11500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:06.184873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41935 (local address 127.31.87.131:51781)
0808 22:32:06.188087 (+ 3214us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:06.188317 (+ 230us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3089,"thread_start_us":202,"threads_started":1}
I20260808 22:32:06.188446 11503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:06.185879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41935 (local address 127.31.87.129:46917)
0808 22:32:06.188059 (+ 2180us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:06.188280 (+ 221us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2028,"thread_start_us":166,"threads_started":1}
W20260808 22:32:06.189158 11183 heartbeater.cc:644] Failed to heartbeat to 127.31.87.190:41935 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.31.87.190:41935: connect: Connection refused (error 111)
W20260808 22:32:06.189515 11401 heartbeater.cc:644] Failed to heartbeat to 127.31.87.190:41935 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.31.87.190:41935: connect: Connection refused (error 111)
W20260808 22:32:06.189497 10966 heartbeater.cc:644] Failed to heartbeat to 127.31.87.190:41935 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.31.87.190:41935: connect: Connection refused (error 111)
I20260808 22:32:06.694502 11505 raft_consensus.cc:482] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7ecff71ab1744a56b7959dc9072f4b64)
I20260808 22:32:06.694726 11505 raft_consensus.cc:504] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "396475d452f242d59b60b887d7c332f0" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37451 } attrs { promote: false } }
I20260808 22:32:06.695032 11506 raft_consensus.cc:482] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7ecff71ab1744a56b7959dc9072f4b64)
I20260808 22:32:06.695209 11506 raft_consensus.cc:504] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "396475d452f242d59b60b887d7c332f0" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37451 } attrs { promote: false } }
I20260808 22:32:06.695820 11505 leader_election.cc:288] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7ecff71ab1744a56b7959dc9072f4b64 (127.31.87.190:41935), 396475d452f242d59b60b887d7c332f0 (127.31.87.188:37451)
I20260808 22:32:06.696134 11506 leader_election.cc:288] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7ecff71ab1744a56b7959dc9072f4b64 (127.31.87.190:41935), f9dc0652a85f416a9c61d82088095a21 (127.31.87.189:34383)
I20260808 22:32:06.696503 11508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:06.695955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41935 (local address 127.0.0.1:47196)
0808 22:32:06.696305 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:06.696404 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":73,"threads_started":1}
W20260808 22:32:06.696707 10345 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41935: connect: Connection refused (error 111)
I20260808 22:32:06.697041 11511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:06.696305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41935 (local address 127.0.0.1:47210)
0808 22:32:06.696871 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:06.696944 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":80,"threads_started":1}
W20260808 22:32:06.697324 10645 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41935: connect: Connection refused (error 111)
I20260808 22:32:06.697606 11508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:06.697359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41935 (local address 127.0.0.1:47218)
0808 22:32:06.697452 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:06.697537 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 22:32:06.697820 10345 leader_election.cc:334] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7ecff71ab1744a56b7959dc9072f4b64 (127.31.87.190:41935): Network error: Client connection negotiation failed: client connection to 127.31.87.190:41935: connect: Connection refused (error 111)
I20260808 22:32:06.698346 11511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:06.698124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41935 (local address 127.0.0.1:47230)
0808 22:32:06.698230 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:06.698281 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260808 22:32:06.698969 10645 leader_election.cc:334] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7ecff71ab1744a56b7959dc9072f4b64 (127.31.87.190:41935): Network error: Client connection negotiation failed: client connection to 127.31.87.190:41935: connect: Connection refused (error 111)
I20260808 22:32:06.700488 11509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:06.696016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.188:37451 (local address 127.0.0.1:42156)
0808 22:32:06.696372 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:06.696387 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:06.696487 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:06.697150 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:06.697153 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:06.697175 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:32:06.697288 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:06.697294 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:06.698351 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:06.698354 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:06.699509 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:06.699520 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:06.700143 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:06.700146 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:06.700212 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:06.700286 (+ 74us) client_negotiation.cc:769] Sending connection context
0808 22:32:06.700340 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 22:32:06.700396 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"mutex_wait_us":31,"thread_start_us":78,"threads_started":1}
I20260808 22:32:06.700554 11515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:06.696561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42156 (local address 127.31.87.188:37451)
0808 22:32:06.696848 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:06.696852 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:06.696866 (+ 14us) server_negotiation.cc:407] Connection header received
0808 22:32:06.696915 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:06.696918 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:06.696963 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:06.697403 (+ 440us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:32:06.697509 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:06.698259 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:06.699637 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:06.700033 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:06.700132 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:06.700194 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:06.700373 (+ 179us) server_negotiation.cc:299] Negotiation successful
0808 22:32:06.700443 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":74,"threads_started":1}
I20260808 22:32:06.700924 10682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f9dc0652a85f416a9c61d82088095a21" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "396475d452f242d59b60b887d7c332f0" is_pre_election: true
I20260808 22:32:06.701169 11513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:06.696327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.189:34383 (local address 127.0.0.1:40192)
0808 22:32:06.697146 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:06.697161 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:06.697491 (+ 330us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:06.698042 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:06.698045 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:06.698062 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:32:06.698153 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:06.698160 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:06.698941 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:06.698945 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:06.700003 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:06.700013 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:06.700828 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:06.700831 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:06.700898 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:06.700917 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 22:32:06.700975 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 22:32:06.701028 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":736,"thread_start_us":80,"threads_started":1}
I20260808 22:32:06.701151 10682 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9dc0652a85f416a9c61d82088095a21 in term 1.
I20260808 22:32:06.701221 11516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:06.696743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40192 (local address 127.31.87.189:34383)
0808 22:32:06.696966 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:06.696970 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:06.697221 (+ 251us) server_negotiation.cc:407] Connection header received
0808 22:32:06.697611 (+ 390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:06.697614 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:06.697655 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:06.697710 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:32:06.698286 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:06.698856 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:06.700132 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:06.700710 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:06.700813 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:06.700874 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:06.701005 (+ 131us) server_negotiation.cc:299] Negotiation successful
0808 22:32:06.701077 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":15,"server-negotiator.queue_time_us":167,"thread_start_us":53,"threads_started":1}
I20260808 22:32:06.701509 10346 leader_election.cc:302] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 396475d452f242d59b60b887d7c332f0, f9dc0652a85f416a9c61d82088095a21; no voters: 7ecff71ab1744a56b7959dc9072f4b64
I20260808 22:32:06.701718 11505 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 22:32:06.701778 11505 raft_consensus.cc:482] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7ecff71ab1744a56b7959dc9072f4b64)
I20260808 22:32:06.701818 11505 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21 [term 1 FOLLOWER]: Advancing to term 2
I20260808 22:32:06.702051 10374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "396475d452f242d59b60b887d7c332f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f9dc0652a85f416a9c61d82088095a21" is_pre_election: true
I20260808 22:32:06.702900 11505 raft_consensus.cc:504] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "396475d452f242d59b60b887d7c332f0" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37451 } attrs { promote: false } }
I20260808 22:32:06.703119 10374 raft_consensus.cc:2379] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 396475d452f242d59b60b887d7c332f0 in current term 2: Already voted for candidate f9dc0652a85f416a9c61d82088095a21 in this term.
I20260808 22:32:06.703243 11505 leader_election.cc:288] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21 [CANDIDATE]: Term 2 election: Requested vote from peers 7ecff71ab1744a56b7959dc9072f4b64 (127.31.87.190:41935), 396475d452f242d59b60b887d7c332f0 (127.31.87.188:37451)
I20260808 22:32:06.703598 11509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:06.703334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41935 (local address 127.0.0.1:47232)
0808 22:32:06.703493 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:06.703541 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260808 22:32:06.703554 10682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f9dc0652a85f416a9c61d82088095a21" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "396475d452f242d59b60b887d7c332f0"
I20260808 22:32:06.703670 10682 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0 [term 1 FOLLOWER]: Advancing to term 2
I20260808 22:32:06.704003 10643 leader_election.cc:302] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 396475d452f242d59b60b887d7c332f0; no voters: 7ecff71ab1744a56b7959dc9072f4b64, f9dc0652a85f416a9c61d82088095a21
I20260808 22:32:06.704209 11509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:06.704007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41935 (local address 127.0.0.1:47238)
0808 22:32:06.704104 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:06.704154 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 22:32:06.704290 10345 leader_election.cc:334] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7ecff71ab1744a56b7959dc9072f4b64 (127.31.87.190:41935): Network error: Client connection negotiation failed: client connection to 127.31.87.190:41935: connect: Connection refused (error 111)
I20260808 22:32:06.704799 10682 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9dc0652a85f416a9c61d82088095a21 in term 2.
I20260808 22:32:06.704903 11506 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 22:32:06.705049 10346 leader_election.cc:302] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 396475d452f242d59b60b887d7c332f0, f9dc0652a85f416a9c61d82088095a21; no voters: 7ecff71ab1744a56b7959dc9072f4b64
I20260808 22:32:06.705235 11505 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 22:32:06.705418 11505 raft_consensus.cc:686] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21 [term 2 LEADER]: Becoming Leader. State: Replica: f9dc0652a85f416a9c61d82088095a21, State: Running, Role: LEADER
I20260808 22:32:06.705634 11505 consensus_queue.cc:237] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "396475d452f242d59b60b887d7c332f0" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37451 } attrs { promote: false } }
I20260808 22:32:06.706735 11522 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f9dc0652a85f416a9c61d82088095a21. Latest consensus state: current_term: 2 leader_uuid: "f9dc0652a85f416a9c61d82088095a21" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "396475d452f242d59b60b887d7c332f0" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37451 } attrs { promote: false } } }
I20260808 22:32:06.706851 11522 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21 [sys.catalog]: This master's current role is: LEADER
I20260808 22:32:06.707119 11525 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:32:06.709393 11525 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:32:06.709769 11525 catalog_manager.cc:1259] Loaded cluster ID: 50f463a19cad4af0a0f0aba6c2c7cfdd
I20260808 22:32:06.709811 11525 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:32:06.710332 11525 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:32:06.710703 11525 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21: Loaded TSK: 0
I20260808 22:32:06.711331 11525 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:32:06.745442 10355 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40190:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:32:06.748273 10355 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:32:06.750988 11509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:06.750786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41935 (local address 127.0.0.1:47266)
0808 22:32:06.750886 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:06.750929 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 22:32:06.751057 10682 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0 [term 2 FOLLOWER]: Refusing update from remote peer f9dc0652a85f416a9c61d82088095a21: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260808 22:32:06.751516 11522 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "396475d452f242d59b60b887d7c332f0" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37451 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260808 22:32:06.751641 11509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:06.751427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41935 (local address 127.0.0.1:47278)
0808 22:32:06.751525 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:06.751570 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 22:32:06.751788 10345 consensus_peers.cc:487] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21 -> Peer 7ecff71ab1744a56b7959dc9072f4b64 (127.31.87.190:41935): Couldn't send request to peer 7ecff71ab1744a56b7959dc9072f4b64. Status: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41935: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 22:32:06.752808 11506 sys_catalog.cc:455] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f9dc0652a85f416a9c61d82088095a21. Latest consensus state: current_term: 2 leader_uuid: "f9dc0652a85f416a9c61d82088095a21" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "396475d452f242d59b60b887d7c332f0" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37451 } attrs { promote: false } } }
I20260808 22:32:06.752925 11506 sys_catalog.cc:458] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:32:06.753908 11522 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "f9dc0652a85f416a9c61d82088095a21" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "396475d452f242d59b60b887d7c332f0" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37451 } attrs { promote: false } } }
I20260808 22:32:06.754011 11522 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21 [sys.catalog]: This master's current role is: LEADER
I20260808 22:32:06.754343 11506 sys_catalog.cc:455] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "f9dc0652a85f416a9c61d82088095a21" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7ecff71ab1744a56b7959dc9072f4b64" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41935 } } peers { permanent_uuid: "f9dc0652a85f416a9c61d82088095a21" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "396475d452f242d59b60b887d7c332f0" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37451 } attrs { promote: false } } }
I20260808 22:32:06.754444 11506 sys_catalog.cc:458] T 00000000000000000000000000000000 P 396475d452f242d59b60b887d7c332f0 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:32:06.763311 11508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:06.759077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:44881 (local address 127.0.0.1:56912)
0808 22:32:06.759207 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:06.759219 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:06.759302 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:06.759681 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:06.759685 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:06.759703 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:32:06.759813 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:06.759820 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:06.761337 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:06.761340 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:06.762147 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:06.762156 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:06.762968 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:06.762972 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:06.763058 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:06.763080 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 22:32:06.763144 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 22:32:06.763206 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260808 22:32:06.763506 11535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:06.759207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56912 (local address 127.31.87.129:44881)
0808 22:32:06.759448 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:06.759453 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:06.759467 (+ 14us) server_negotiation.cc:407] Connection header received
0808 22:32:06.759517 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:06.759520 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:06.759565 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:06.759926 (+ 361us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:32:06.760044 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:06.761241 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:06.762289 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:06.762863 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:06.762957 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:06.763018 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:06.763167 (+ 149us) server_negotiation.cc:299] Negotiation successful
0808 22:32:06.763239 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":71,"threads_started":1}
I20260808 22:32:06.763660 11509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:06.758841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.131:46167 (local address 127.0.0.1:54194)
0808 22:32:06.758941 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:06.758955 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:06.759028 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:06.760066 (+ 1038us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:06.760070 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:06.760085 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:32:06.760151 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:06.760156 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:06.761191 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:06.761195 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:06.762412 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:06.762419 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:06.763296 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:06.763300 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:06.763402 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:06.763422 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 22:32:06.763475 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 22:32:06.763541 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":17}
I20260808 22:32:06.763693 11537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:06.759321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54194 (local address 127.31.87.131:46167)
0808 22:32:06.759566 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:06.759571 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:06.759583 (+ 12us) server_negotiation.cc:407] Connection header received
0808 22:32:06.759634 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:06.759636 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:06.759680 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:06.760247 (+ 567us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:32:06.760375 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:06.761078 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:06.762549 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:06.763176 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:06.763421 (+ 245us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:06.763480 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:06.763513 (+ 33us) server_negotiation.cc:299] Negotiation successful
0808 22:32:06.763579 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":72,"threads_started":1}
I20260808 22:32:06.763957 10855 tablet_service.cc:1467] Processing CreateTablet for tablet 51e6d2bedf81440f8dde92125a253c51 (DEFAULT_TABLE table=test-table [id=49393109d111448c96ea698042373bc1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:32:06.764143 11539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:06.759330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.130:35045 (local address 127.0.0.1:54834)
0808 22:32:06.759869 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:06.759882 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:06.759951 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:06.760286 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:06.760289 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:06.760306 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:32:06.760389 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:06.760395 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:06.761251 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:06.761254 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:06.762405 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:06.762416 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:06.763371 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:06.763374 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:06.763860 (+ 486us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:06.763883 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 22:32:06.763961 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 22:32:06.764026 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"mutex_wait_us":43,"thread_start_us":85,"threads_started":1}
I20260808 22:32:06.764247 10855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51e6d2bedf81440f8dde92125a253c51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:32:06.764268 11299 tablet_service.cc:1467] Processing CreateTablet for tablet 51e6d2bedf81440f8dde92125a253c51 (DEFAULT_TABLE table=test-table [id=49393109d111448c96ea698042373bc1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:32:06.764412 11541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:06.759667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54834 (local address 127.31.87.130:35045)
0808 22:32:06.759910 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:06.759914 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:06.759928 (+ 14us) server_negotiation.cc:407] Connection header received
0808 22:32:06.760037 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:06.760040 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:06.760088 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:06.760169 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:32:06.760522 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:06.761134 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:06.762549 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:06.763163 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:06.763266 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:06.763323 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:06.764257 (+ 934us) server_negotiation.cc:299] Negotiation successful
0808 22:32:06.764320 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":72,"threads_started":1}
I20260808 22:32:06.764494 11299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51e6d2bedf81440f8dde92125a253c51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:32:06.765554 11073 tablet_service.cc:1467] Processing CreateTablet for tablet 51e6d2bedf81440f8dde92125a253c51 (DEFAULT_TABLE table=test-table [id=49393109d111448c96ea698042373bc1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:32:06.765810 11073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51e6d2bedf81440f8dde92125a253c51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:32:06.766793 11543 tablet_bootstrap.cc:492] T 51e6d2bedf81440f8dde92125a253c51 P 2d486908e4e54ccbb78867ae276bf39f: Bootstrap starting.
I20260808 22:32:06.767297 11545 tablet_bootstrap.cc:492] T 51e6d2bedf81440f8dde92125a253c51 P 80ab594182ec4c6c95579360d4cc7210: Bootstrap starting.
I20260808 22:32:06.767836 11543 tablet_bootstrap.cc:654] T 51e6d2bedf81440f8dde92125a253c51 P 2d486908e4e54ccbb78867ae276bf39f: Neither blocks nor log segments found. Creating new log.
I20260808 22:32:06.768105 11545 tablet_bootstrap.cc:654] T 51e6d2bedf81440f8dde92125a253c51 P 80ab594182ec4c6c95579360d4cc7210: Neither blocks nor log segments found. Creating new log.
I20260808 22:32:06.768707 11546 tablet_bootstrap.cc:492] T 51e6d2bedf81440f8dde92125a253c51 P 0490e64eb57d43268a8e1c61d601e835: Bootstrap starting.
I20260808 22:32:06.768939 11543 tablet_bootstrap.cc:492] T 51e6d2bedf81440f8dde92125a253c51 P 2d486908e4e54ccbb78867ae276bf39f: No bootstrap required, opened a new log
I20260808 22:32:06.769021 11543 ts_tablet_manager.cc:1397] T 51e6d2bedf81440f8dde92125a253c51 P 2d486908e4e54ccbb78867ae276bf39f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 22:32:06.769198 11545 tablet_bootstrap.cc:492] T 51e6d2bedf81440f8dde92125a253c51 P 80ab594182ec4c6c95579360d4cc7210: No bootstrap required, opened a new log
I20260808 22:32:06.769294 11545 ts_tablet_manager.cc:1397] T 51e6d2bedf81440f8dde92125a253c51 P 80ab594182ec4c6c95579360d4cc7210: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 22:32:06.769515 11543 raft_consensus.cc:348] T 51e6d2bedf81440f8dde92125a253c51 P 2d486908e4e54ccbb78867ae276bf39f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80ab594182ec4c6c95579360d4cc7210" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 46167 } } peers { permanent_uuid: "2d486908e4e54ccbb78867ae276bf39f" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 44881 } } peers { permanent_uuid: "0490e64eb57d43268a8e1c61d601e835" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 35045 } }
I20260808 22:32:06.769637 11543 raft_consensus.cc:374] T 51e6d2bedf81440f8dde92125a253c51 P 2d486908e4e54ccbb78867ae276bf39f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:32:06.769675 11543 raft_consensus.cc:729] T 51e6d2bedf81440f8dde92125a253c51 P 2d486908e4e54ccbb78867ae276bf39f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d486908e4e54ccbb78867ae276bf39f, State: Initialized, Role: FOLLOWER
I20260808 22:32:06.769631 11545 raft_consensus.cc:348] T 51e6d2bedf81440f8dde92125a253c51 P 80ab594182ec4c6c95579360d4cc7210 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80ab594182ec4c6c95579360d4cc7210" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 46167 } } peers { permanent_uuid: "2d486908e4e54ccbb78867ae276bf39f" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 44881 } } peers { permanent_uuid: "0490e64eb57d43268a8e1c61d601e835" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 35045 } }
I20260808 22:32:06.769749 11545 raft_consensus.cc:374] T 51e6d2bedf81440f8dde92125a253c51 P 80ab594182ec4c6c95579360d4cc7210 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:32:06.769783 11545 raft_consensus.cc:729] T 51e6d2bedf81440f8dde92125a253c51 P 80ab594182ec4c6c95579360d4cc7210 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80ab594182ec4c6c95579360d4cc7210, State: Initialized, Role: FOLLOWER
I20260808 22:32:06.769851 11543 consensus_queue.cc:260] T 51e6d2bedf81440f8dde92125a253c51 P 2d486908e4e54ccbb78867ae276bf39f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80ab594182ec4c6c95579360d4cc7210" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 46167 } } peers { permanent_uuid: "2d486908e4e54ccbb78867ae276bf39f" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 44881 } } peers { permanent_uuid: "0490e64eb57d43268a8e1c61d601e835" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 35045 } }
I20260808 22:32:06.769901 11545 consensus_queue.cc:260] T 51e6d2bedf81440f8dde92125a253c51 P 80ab594182ec4c6c95579360d4cc7210 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80ab594182ec4c6c95579360d4cc7210" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 46167 } } peers { permanent_uuid: "2d486908e4e54ccbb78867ae276bf39f" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 44881 } } peers { permanent_uuid: "0490e64eb57d43268a8e1c61d601e835" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 35045 } }
I20260808 22:32:06.770200 11545 ts_tablet_manager.cc:1428] T 51e6d2bedf81440f8dde92125a253c51 P 80ab594182ec4c6c95579360d4cc7210: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 22:32:06.771248 11560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:06.770861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41935 (local address 127.31.87.131:51365)
0808 22:32:06.771098 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:06.771155 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":92,"threads_started":1}
I20260808 22:32:06.772311 11560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:06.772089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41935 (local address 127.31.87.131:40287)
0808 22:32:06.772199 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:06.772253 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 22:32:06.772428 11546 tablet_bootstrap.cc:654] T 51e6d2bedf81440f8dde92125a253c51 P 0490e64eb57d43268a8e1c61d601e835: Neither blocks nor log segments found. Creating new log.
I20260808 22:32:06.773589 11543 ts_tablet_manager.cc:1428] T 51e6d2bedf81440f8dde92125a253c51 P 2d486908e4e54ccbb78867ae276bf39f: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260808 22:32:06.773644 11404 heartbeater.cc:503] Master 127.31.87.189:34383 was elected leader, sending a full tablet report...
I20260808 22:32:06.773698 11546 tablet_bootstrap.cc:492] T 51e6d2bedf81440f8dde92125a253c51 P 0490e64eb57d43268a8e1c61d601e835: No bootstrap required, opened a new log
I20260808 22:32:06.773778 11546 ts_tablet_manager.cc:1397] T 51e6d2bedf81440f8dde92125a253c51 P 0490e64eb57d43268a8e1c61d601e835: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260808 22:32:06.773795 11559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:06.770462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41935 (local address 127.31.87.129:50559)
0808 22:32:06.771620 (+ 1158us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:06.773699 (+ 2079us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1098,"thread_start_us":111,"threads_started":1}
I20260808 22:32:06.773957 10967 heartbeater.cc:503] Master 127.31.87.189:34383 was elected leader, sending a full tablet report...
I20260808 22:32:06.774156 11546 raft_consensus.cc:348] T 51e6d2bedf81440f8dde92125a253c51 P 0490e64eb57d43268a8e1c61d601e835 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80ab594182ec4c6c95579360d4cc7210" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 46167 } } peers { permanent_uuid: "2d486908e4e54ccbb78867ae276bf39f" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 44881 } } peers { permanent_uuid: "0490e64eb57d43268a8e1c61d601e835" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 35045 } }
I20260808 22:32:06.774282 11546 raft_consensus.cc:374] T 51e6d2bedf81440f8dde92125a253c51 P 0490e64eb57d43268a8e1c61d601e835 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:32:06.774323 11546 raft_consensus.cc:729] T 51e6d2bedf81440f8dde92125a253c51 P 0490e64eb57d43268a8e1c61d601e835 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0490e64eb57d43268a8e1c61d601e835, State: Initialized, Role: FOLLOWER
I20260808 22:32:06.774448 11546 consensus_queue.cc:260] T 51e6d2bedf81440f8dde92125a253c51 P 0490e64eb57d43268a8e1c61d601e835 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80ab594182ec4c6c95579360d4cc7210" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 46167 } } peers { permanent_uuid: "2d486908e4e54ccbb78867ae276bf39f" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 44881 } } peers { permanent_uuid: "0490e64eb57d43268a8e1c61d601e835" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 35045 } }
I20260808 22:32:06.774632 11559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:06.774242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41935 (local address 127.31.87.129:35761)
0808 22:32:06.774451 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:06.774502 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260808 22:32:06.774756 11546 ts_tablet_manager.cc:1428] T 51e6d2bedf81440f8dde92125a253c51 P 0490e64eb57d43268a8e1c61d601e835: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 22:32:06.776137 11565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:06.775748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41935 (local address 127.31.87.130:48733)
0808 22:32:06.776003 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:06.776057 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":95,"threads_started":1}
I20260808 22:32:06.776721 11565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:06.776522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41935 (local address 127.31.87.130:41073)
0808 22:32:06.776615 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:06.776662 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 22:32:06.776976 11187 heartbeater.cc:503] Master 127.31.87.189:34383 was elected leader, sending a full tablet report...
I20260808 22:32:07.033365 11551 raft_consensus.cc:482] T 51e6d2bedf81440f8dde92125a253c51 P 2d486908e4e54ccbb78867ae276bf39f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:32:07.033505 11551 raft_consensus.cc:504] T 51e6d2bedf81440f8dde92125a253c51 P 2d486908e4e54ccbb78867ae276bf39f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80ab594182ec4c6c95579360d4cc7210" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 46167 } } peers { permanent_uuid: "2d486908e4e54ccbb78867ae276bf39f" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 44881 } } peers { permanent_uuid: "0490e64eb57d43268a8e1c61d601e835" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 35045 } }
I20260808 22:32:07.034571 11551 leader_election.cc:288] T 51e6d2bedf81440f8dde92125a253c51 P 2d486908e4e54ccbb78867ae276bf39f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80ab594182ec4c6c95579360d4cc7210 (127.31.87.131:46167), 0490e64eb57d43268a8e1c61d601e835 (127.31.87.130:35045)
I20260808 22:32:07.039394 11541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:07.035228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:59547 (local address 127.31.87.130:35045)
0808 22:32:07.035356 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:07.035360 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:07.035473 (+ 113us) server_negotiation.cc:407] Connection header received
0808 22:32:07.035583 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:07.035587 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:07.035638 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:07.035710 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:32:07.035953 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:07.036698 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:07.038126 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:07.038748 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:07.038854 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:07.038906 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:07.039196 (+ 290us) server_negotiation.cc:299] Negotiation successful
0808 22:32:07.039265 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 22:32:07.039394 11537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:07.034927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:35655 (local address 127.31.87.131:46167)
0808 22:32:07.035053 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:07.035058 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:07.035073 (+ 15us) server_negotiation.cc:407] Connection header received
0808 22:32:07.035229 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:07.035233 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:07.035300 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:07.035378 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:32:07.035709 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:07.036499 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:07.038128 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:07.038733 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:07.038854 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:07.038906 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:07.039196 (+ 290us) server_negotiation.cc:299] Negotiation successful
0808 22:32:07.039265 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 22:32:07.039425 11559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:07.034815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.131:46167 (local address 127.31.87.129:35655)
0808 22:32:07.034957 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:07.034974 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:07.035099 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:07.035489 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:07.035492 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:07.035509 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:32:07.035602 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:07.035606 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:07.036624 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:07.036627 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:07.038003 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:07.038014 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:07.038874 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:07.038878 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:07.039049 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:07.039098 (+ 49us) client_negotiation.cc:769] Sending connection context
0808 22:32:07.039167 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 22:32:07.039275 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":63}
I20260808 22:32:07.039433 11567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:07.035103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.130:35045 (local address 127.31.87.129:59547)
0808 22:32:07.035395 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:07.035407 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:07.035500 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:07.035736 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:07.035738 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:07.035774 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:32:07.035860 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:07.035865 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:07.036805 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:07.036808 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:07.038009 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:07.038016 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:07.038874 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:07.038878 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:07.039074 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:07.039109 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 22:32:07.039171 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 22:32:07.039266 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"mutex_wait_us":154,"thread_start_us":133,"threads_started":1}
I20260808 22:32:07.039893 11333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e6d2bedf81440f8dde92125a253c51" candidate_uuid: "2d486908e4e54ccbb78867ae276bf39f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80ab594182ec4c6c95579360d4cc7210" is_pre_election: true
I20260808 22:32:07.040045 11333 raft_consensus.cc:2454] T 51e6d2bedf81440f8dde92125a253c51 P 80ab594182ec4c6c95579360d4cc7210 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d486908e4e54ccbb78867ae276bf39f in term 0.
I20260808 22:32:07.040107 11103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e6d2bedf81440f8dde92125a253c51" candidate_uuid: "2d486908e4e54ccbb78867ae276bf39f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0490e64eb57d43268a8e1c61d601e835" is_pre_election: true
I20260808 22:32:07.040333 11103 raft_consensus.cc:2454] T 51e6d2bedf81440f8dde92125a253c51 P 0490e64eb57d43268a8e1c61d601e835 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d486908e4e54ccbb78867ae276bf39f in term 0.
I20260808 22:32:07.040413 10783 leader_election.cc:302] T 51e6d2bedf81440f8dde92125a253c51 P 2d486908e4e54ccbb78867ae276bf39f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d486908e4e54ccbb78867ae276bf39f, 80ab594182ec4c6c95579360d4cc7210; no voters:
I20260808 22:32:07.040704 11551 raft_consensus.cc:2781] T 51e6d2bedf81440f8dde92125a253c51 P 2d486908e4e54ccbb78867ae276bf39f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:32:07.040760 11551 raft_consensus.cc:482] T 51e6d2bedf81440f8dde92125a253c51 P 2d486908e4e54ccbb78867ae276bf39f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:32:07.040788 11551 raft_consensus.cc:3037] T 51e6d2bedf81440f8dde92125a253c51 P 2d486908e4e54ccbb78867ae276bf39f [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:32:07.041795 11551 raft_consensus.cc:504] T 51e6d2bedf81440f8dde92125a253c51 P 2d486908e4e54ccbb78867ae276bf39f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80ab594182ec4c6c95579360d4cc7210" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 46167 } } peers { permanent_uuid: "2d486908e4e54ccbb78867ae276bf39f" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 44881 } } peers { permanent_uuid: "0490e64eb57d43268a8e1c61d601e835" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 35045 } }
I20260808 22:32:07.042090 11551 leader_election.cc:288] T 51e6d2bedf81440f8dde92125a253c51 P 2d486908e4e54ccbb78867ae276bf39f [CANDIDATE]: Term 1 election: Requested vote from peers 80ab594182ec4c6c95579360d4cc7210 (127.31.87.131:46167), 0490e64eb57d43268a8e1c61d601e835 (127.31.87.130:35045)
I20260808 22:32:07.042387 11333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e6d2bedf81440f8dde92125a253c51" candidate_uuid: "2d486908e4e54ccbb78867ae276bf39f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80ab594182ec4c6c95579360d4cc7210"
I20260808 22:32:07.042496 11333 raft_consensus.cc:3037] T 51e6d2bedf81440f8dde92125a253c51 P 80ab594182ec4c6c95579360d4cc7210 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:32:07.042462 11103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e6d2bedf81440f8dde92125a253c51" candidate_uuid: "2d486908e4e54ccbb78867ae276bf39f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0490e64eb57d43268a8e1c61d601e835"
I20260808 22:32:07.042579 11103 raft_consensus.cc:3037] T 51e6d2bedf81440f8dde92125a253c51 P 0490e64eb57d43268a8e1c61d601e835 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:32:07.043342 11333 raft_consensus.cc:2454] T 51e6d2bedf81440f8dde92125a253c51 P 80ab594182ec4c6c95579360d4cc7210 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d486908e4e54ccbb78867ae276bf39f in term 1.
I20260808 22:32:07.043529 11103 raft_consensus.cc:2454] T 51e6d2bedf81440f8dde92125a253c51 P 0490e64eb57d43268a8e1c61d601e835 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d486908e4e54ccbb78867ae276bf39f in term 1.
I20260808 22:32:07.043610 10783 leader_election.cc:302] T 51e6d2bedf81440f8dde92125a253c51 P 2d486908e4e54ccbb78867ae276bf39f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d486908e4e54ccbb78867ae276bf39f, 80ab594182ec4c6c95579360d4cc7210; no voters:
I20260808 22:32:07.043756 11551 raft_consensus.cc:2781] T 51e6d2bedf81440f8dde92125a253c51 P 2d486908e4e54ccbb78867ae276bf39f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:32:07.043963 11551 raft_consensus.cc:686] T 51e6d2bedf81440f8dde92125a253c51 P 2d486908e4e54ccbb78867ae276bf39f [term 1 LEADER]: Becoming Leader. State: Replica: 2d486908e4e54ccbb78867ae276bf39f, State: Running, Role: LEADER
I20260808 22:32:07.044189 11551 consensus_queue.cc:237] T 51e6d2bedf81440f8dde92125a253c51 P 2d486908e4e54ccbb78867ae276bf39f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80ab594182ec4c6c95579360d4cc7210" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 46167 } } peers { permanent_uuid: "2d486908e4e54ccbb78867ae276bf39f" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 44881 } } peers { permanent_uuid: "0490e64eb57d43268a8e1c61d601e835" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 35045 } }
I20260808 22:32:07.044747 11559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:07.044522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41935 (local address 127.31.87.129:43111)
0808 22:32:07.044623 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:07.044679 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 22:32:07.046384 10355 catalog_manager.cc:5495] T 51e6d2bedf81440f8dde92125a253c51 P 2d486908e4e54ccbb78867ae276bf39f reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d486908e4e54ccbb78867ae276bf39f (127.31.87.129). New cstate: current_term: 1 leader_uuid: "2d486908e4e54ccbb78867ae276bf39f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80ab594182ec4c6c95579360d4cc7210" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 46167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d486908e4e54ccbb78867ae276bf39f" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 44881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0490e64eb57d43268a8e1c61d601e835" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 35045 } health_report { overall_health: UNKNOWN } } }
I20260808 22:32:07.124357 11535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:07.121035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56924 (local address 127.31.87.129:44881)
0808 22:32:07.121183 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:07.121188 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:07.121203 (+ 15us) server_negotiation.cc:407] Connection header received
0808 22:32:07.121304 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:07.121307 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:07.121363 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:07.121453 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:32:07.121695 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:07.122186 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:07.123018 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:07.123226 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:07.123322 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:07.123566 (+ 244us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:32:07.123984 (+ 418us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:32:07.124084 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:07.124161 (+ 77us) server_negotiation.cc:299] Negotiation successful
0808 22:32:07.124216 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 22:32:07.127198 11333 raft_consensus.cc:1261] T 51e6d2bedf81440f8dde92125a253c51 P 80ab594182ec4c6c95579360d4cc7210 [term 1 FOLLOWER]: Refusing update from remote peer 2d486908e4e54ccbb78867ae276bf39f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:32:07.127442 11103 raft_consensus.cc:1261] T 51e6d2bedf81440f8dde92125a253c51 P 0490e64eb57d43268a8e1c61d601e835 [term 1 FOLLOWER]: Refusing update from remote peer 2d486908e4e54ccbb78867ae276bf39f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:32:07.127743 11569 consensus_queue.cc:1035] T 51e6d2bedf81440f8dde92125a253c51 P 2d486908e4e54ccbb78867ae276bf39f [LEADER]: Connected to new peer: Peer: permanent_uuid: "80ab594182ec4c6c95579360d4cc7210" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 46167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:32:07.127974 11551 consensus_queue.cc:1035] T 51e6d2bedf81440f8dde92125a253c51 P 2d486908e4e54ccbb78867ae276bf39f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0490e64eb57d43268a8e1c61d601e835" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 35045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:32:07.129674 11565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:07.129379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41935 (local address 127.31.87.130:56103)
0808 22:32:07.129529 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:07.129587 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260808 22:32:07.130394 11560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:07.129343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41935 (local address 127.31.87.131:49649)
0808 22:32:07.130269 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:07.130324 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":860}
I20260808 22:32:07.132184 11559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:07.131915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41935 (local address 127.31.87.129:50499)
0808 22:32:07.132066 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:07.132115 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260808 22:32:07.133282 11565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:07.133019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41935 (local address 127.31.87.130:48139)
0808 22:32:07.133159 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:07.133211 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 22:32:07.135525 11559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:07.135048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41935 (local address 127.31.87.129:46247)
0808 22:32:07.135406 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:07.135462 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":293}
I20260808 22:32:07.137627 11560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:07.137366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41935 (local address 127.31.87.131:45235)
0808 22:32:07.137503 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:07.137558 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260808 22:32:07.140175 10360 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40190:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:32:07.140321 10360 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40190:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:32:07.143656 10360 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f9dc0652a85f416a9c61d82088095a21: Sending DeleteTablet for 3 replicas of tablet 51e6d2bedf81440f8dde92125a253c51
I20260808 22:32:07.144392 11073 tablet_service.cc:1514] Processing DeleteTablet for tablet 51e6d2bedf81440f8dde92125a253c51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:32:07 UTC) from {username='oryx', principal='oryx/127.31.87.189@KRBTEST.COM'} at 127.0.0.1:54834
I20260808 22:32:07.144680 11299 tablet_service.cc:1514] Processing DeleteTablet for tablet 51e6d2bedf81440f8dde92125a253c51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:32:07 UTC) from {username='oryx', principal='oryx/127.31.87.189@KRBTEST.COM'} at 127.0.0.1:54194
I20260808 22:32:07.144711 11589 tablet_replica.cc:330] T 51e6d2bedf81440f8dde92125a253c51 P 0490e64eb57d43268a8e1c61d601e835: stopping tablet replica
I20260808 22:32:07.144842 11589 raft_consensus.cc:2229] T 51e6d2bedf81440f8dde92125a253c51 P 0490e64eb57d43268a8e1c61d601e835 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:32:07.144944 11590 tablet_replica.cc:330] T 51e6d2bedf81440f8dde92125a253c51 P 80ab594182ec4c6c95579360d4cc7210: stopping tablet replica
I20260808 22:32:07.144966 11589 raft_consensus.cc:2258] T 51e6d2bedf81440f8dde92125a253c51 P 0490e64eb57d43268a8e1c61d601e835 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:32:07.145056 11590 raft_consensus.cc:2229] T 51e6d2bedf81440f8dde92125a253c51 P 80ab594182ec4c6c95579360d4cc7210 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:32:07.145205 11590 raft_consensus.cc:2258] T 51e6d2bedf81440f8dde92125a253c51 P 80ab594182ec4c6c95579360d4cc7210 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:32:07.145638 11589 ts_tablet_manager.cc:1905] T 51e6d2bedf81440f8dde92125a253c51 P 0490e64eb57d43268a8e1c61d601e835: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:32:07.145857 11590 ts_tablet_manager.cc:1905] T 51e6d2bedf81440f8dde92125a253c51 P 80ab594182ec4c6c95579360d4cc7210: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:32:07.146116 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 10612
I20260808 22:32:07.149080 11589 ts_tablet_manager.cc:1918] T 51e6d2bedf81440f8dde92125a253c51 P 0490e64eb57d43268a8e1c61d601e835: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:32:07.149155 11589 log.cc:1196] T 51e6d2bedf81440f8dde92125a253c51 P 0490e64eb57d43268a8e1c61d601e835: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-1/wal/wals/51e6d2bedf81440f8dde92125a253c51
I20260808 22:32:07.149154 11590 ts_tablet_manager.cc:1918] T 51e6d2bedf81440f8dde92125a253c51 P 80ab594182ec4c6c95579360d4cc7210: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:32:07.149235 11590 log.cc:1196] T 51e6d2bedf81440f8dde92125a253c51 P 80ab594182ec4c6c95579360d4cc7210: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786228291013016-32094-0/minicluster-data/ts-2/wal/wals/51e6d2bedf81440f8dde92125a253c51
I20260808 22:32:07.149392 11589 ts_tablet_manager.cc:1939] T 51e6d2bedf81440f8dde92125a253c51 P 0490e64eb57d43268a8e1c61d601e835: Deleting consensus metadata
I20260808 22:32:07.149523 11590 ts_tablet_manager.cc:1939] T 51e6d2bedf81440f8dde92125a253c51 P 80ab594182ec4c6c95579360d4cc7210: Deleting consensus metadata
I20260808 22:32:07.149930 10345 catalog_manager.cc:4841] TS 0490e64eb57d43268a8e1c61d601e835 (127.31.87.130:35045): tablet 51e6d2bedf81440f8dde92125a253c51 (table test-table [id=49393109d111448c96ea698042373bc1]) successfully deleted
I20260808 22:32:07.154714 10345 catalog_manager.cc:4841] TS 80ab594182ec4c6c95579360d4cc7210 (127.31.87.131:46167): tablet 51e6d2bedf81440f8dde92125a253c51 (table test-table [id=49393109d111448c96ea698042373bc1]) successfully deleted
W20260808 22:32:07.159921 10345 connection.cc:535] client connection to 127.31.87.129:44881 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 22:32:07.160392 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 10975
I20260808 22:32:07.160637 11539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:07.160399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:44881 (local address 127.0.0.1:56926)
0808 22:32:07.160511 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:07.160557 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.129:44881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 22:32:07.161851 10345 catalog_manager.cc:4584] TS 2d486908e4e54ccbb78867ae276bf39f (127.31.87.129:44881): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 51e6d2bedf81440f8dde92125a253c51: Network error: Client connection negotiation failed: client connection to 127.31.87.129:44881: connect: Connection refused (error 111)
I20260808 22:32:07.170260 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 11199
I20260808 22:32:07.180480 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 10152
I20260808 22:32:07.190414 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 10322
2026-08-08T22:32:07Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5720 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:32:07 dist-test-slave-266d krb5kdc[11600](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:32:07 dist-test-slave-266d krb5kdc[11600](info): set up 2 sockets
Aug 08 22:32:07 dist-test-slave-266d krb5kdc[11600](info): commencing operation
krb5kdc: starting...
W20260808 22:32:08.353849 32094 mini_kdc.cc:121] Time spent starting KDC: real 1.140s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Aug 08 22:32:08 dist-test-slave-266d krb5kdc[11600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:32:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:32:08Z Disabled control of system clock
WARNING: no policy specified for oryx/127.31.87.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.87.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.87.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.190.keytab.
Entry for principal oryx/127.31.87.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.190.keytab.
I20260808 22:32:08.471931 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:45745
--webserver_interface=127.31.87.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.190.keytab
--principal=oryx/127.31.87.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:38895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:45745
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:32:08.582804 11623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:32:08.583069 11623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:32:08.583112 11623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:32:08.587051 11623 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:32:08.587106 11623 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 22:32:08.587126 11623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:32:08.587145 11623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:32:08.587167 11623 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:32:08.587203 11623 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:32:08.592346 11623 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:38895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:45745
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.190.keytab
--principal=oryx/127.31.87.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:45745
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.87.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:32:08.593367 11623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:32:08.594372 11623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:32:08.602787 11639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:08.602800 11634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:08.602800 11636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:08.602855 11623 server_base.cc:1034] running on GCE node
I20260808 22:32:08.603571 11623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:32:08.604620 11623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:32:08.605775 11623 hybrid_clock.cc:648] HybridClock initialized: now 1786228328605766 us; error 30 us; skew 500 ppm
Aug 08 22:32:08 dist-test-slave-266d krb5kdc[11600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228328, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.87.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:32:08.608301 11623 init.cc:377] Logged in from keytab as oryx/127.31.87.190@KRBTEST.COM (short username oryx)
I20260808 22:32:08.609150 11623 webserver.cc:460] Webserver started at http://127.31.87.190:44231/ using document root <none> and password file <none>
I20260808 22:32:08.609684 11623 fs_manager.cc:359] Metadata directory not provided
I20260808 22:32:08.609750 11623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:32:08.609935 11623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:32:08.611886 11623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/master-0/data/instance:
uuid: "9f31f95f36104c6fa687563b1fdb4d24"
format_stamp: "Formatted at 2026-08-08 22:32:08 on dist-test-slave-266d"
server_key: "bb1a10c5940d56301f2691a5d7cf045a"
server_key_iv: "c1098dc6a7212fdf73a6b997df15462b"
server_key_version: "encryptionkey@0"
I20260808 22:32:08.612401 11623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/master-0/wal/instance:
uuid: "9f31f95f36104c6fa687563b1fdb4d24"
format_stamp: "Formatted at 2026-08-08 22:32:08 on dist-test-slave-266d"
server_key: "bb1a10c5940d56301f2691a5d7cf045a"
server_key_iv: "c1098dc6a7212fdf73a6b997df15462b"
server_key_version: "encryptionkey@0"
I20260808 22:32:08.615971 11623 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260808 22:32:08.618286 11649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:08.619383 11623 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:32:08.619494 11623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "9f31f95f36104c6fa687563b1fdb4d24"
format_stamp: "Formatted at 2026-08-08 22:32:08 on dist-test-slave-266d"
server_key: "bb1a10c5940d56301f2691a5d7cf045a"
server_key_iv: "c1098dc6a7212fdf73a6b997df15462b"
server_key_version: "encryptionkey@0"
I20260808 22:32:08.619585 11623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:32:08.629680 11623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:32:08.632176 11623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:32:08.632344 11623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:32:08.639717 11623 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.190:45745
I20260808 22:32:08.639721 11743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.190:45745 every 8 connection(s)
I20260808 22:32:08.640575 11623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:32:08.643146 11744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:32:08.647472 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 11623
I20260808 22:32:08.647605 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/master-0/wal/instance
I20260808 22:32:08.647887 32094 external_mini_cluster.cc:1428] Setting key 91303aefbe277c1a350cbb8ffde52e70
I20260808 22:32:08.648418 11744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f31f95f36104c6fa687563b1fdb4d24: Bootstrap starting.
I20260808 22:32:08.650676 11744 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f31f95f36104c6fa687563b1fdb4d24: Neither blocks nor log segments found. Creating new log.
I20260808 22:32:08.651547 11744 log.cc:824] T 00000000000000000000000000000000 P 9f31f95f36104c6fa687563b1fdb4d24: Log is configured to *not* fsync() on all Append() calls
I20260808 22:32:08.653482 11744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f31f95f36104c6fa687563b1fdb4d24: No bootstrap required, opened a new log
Aug 08 22:32:08 dist-test-slave-266d krb5kdc[11600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM
I20260808 22:32:08.657428 11744 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f31f95f36104c6fa687563b1fdb4d24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f31f95f36104c6fa687563b1fdb4d24" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45745 } }
I20260808 22:32:08.657692 11744 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f31f95f36104c6fa687563b1fdb4d24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:32:08.657755 11744 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f31f95f36104c6fa687563b1fdb4d24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f31f95f36104c6fa687563b1fdb4d24, State: Initialized, Role: FOLLOWER
I20260808 22:32:08.658226 11744 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f31f95f36104c6fa687563b1fdb4d24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f31f95f36104c6fa687563b1fdb4d24" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45745 } }
I20260808 22:32:08.658419 11744 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9f31f95f36104c6fa687563b1fdb4d24 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:32:08.658488 11744 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f31f95f36104c6fa687563b1fdb4d24 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:32:08.658586 11744 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f31f95f36104c6fa687563b1fdb4d24 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:32:08.659586 11744 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f31f95f36104c6fa687563b1fdb4d24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f31f95f36104c6fa687563b1fdb4d24" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45745 } }
I20260808 22:32:08.659963 11744 leader_election.cc:302] T 00000000000000000000000000000000 P 9f31f95f36104c6fa687563b1fdb4d24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f31f95f36104c6fa687563b1fdb4d24; no voters:
I20260808 22:32:08.660303 11744 leader_election.cc:288] T 00000000000000000000000000000000 P 9f31f95f36104c6fa687563b1fdb4d24 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:32:08.660377 11750 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f31f95f36104c6fa687563b1fdb4d24 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:32:08.660589 11750 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f31f95f36104c6fa687563b1fdb4d24 [term 1 LEADER]: Becoming Leader. State: Replica: 9f31f95f36104c6fa687563b1fdb4d24, State: Running, Role: LEADER
I20260808 22:32:08.660940 11750 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f31f95f36104c6fa687563b1fdb4d24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f31f95f36104c6fa687563b1fdb4d24" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45745 } }
I20260808 22:32:08.661646 11744 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f31f95f36104c6fa687563b1fdb4d24 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:32:08.662600 11752 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f31f95f36104c6fa687563b1fdb4d24 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f31f95f36104c6fa687563b1fdb4d24. Latest consensus state: current_term: 1 leader_uuid: "9f31f95f36104c6fa687563b1fdb4d24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f31f95f36104c6fa687563b1fdb4d24" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45745 } } }
I20260808 22:32:08.662720 11752 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f31f95f36104c6fa687563b1fdb4d24 [sys.catalog]: This master's current role is: LEADER
I20260808 22:32:08.662751 11748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:08.649145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33802 (local address 127.31.87.190:45745)
0808 22:32:08.649527 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:08.649541 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:08.649576 (+ 35us) server_negotiation.cc:407] Connection header received
0808 22:32:08.650169 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:08.650185 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:08.650481 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:08.650753 (+ 272us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:08.651588 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:08.652546 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:08.653684 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:08.654146 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:08.654257 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:08.657056 (+ 2799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:08.657062 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:08.657077 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:08.657140 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:08.659118 (+ 1978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:08.659549 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:08.659555 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:08.659561 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:08.659746 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:08.660037 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:08.660040 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:08.660042 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:08.660569 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:08.661357 (+ 788us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:08.661503 (+ 146us) server_negotiation.cc:299] Negotiation successful
0808 22:32:08.661798 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":102,"threads_started":1}
I20260808 22:32:08.663043 11751 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f31f95f36104c6fa687563b1fdb4d24 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9f31f95f36104c6fa687563b1fdb4d24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f31f95f36104c6fa687563b1fdb4d24" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45745 } } }
I20260808 22:32:08.663132 11751 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f31f95f36104c6fa687563b1fdb4d24 [sys.catalog]: This master's current role is: LEADER
W20260808 22:32:08.666232 11777 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9f31f95f36104c6fa687563b1fdb4d24: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:32:08.666296 11777 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:32:08.666393 11763 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:32:08.667068 11763 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:32:08.672281 11763 catalog_manager.cc:1347] Generated new cluster ID: 62669f7642c04e8e86966c7a4e289331
I20260808 22:32:08.672340 11763 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:32:08.683842 11763 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:32:08.685122 11763 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:32:08.697551 11763 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9f31f95f36104c6fa687563b1fdb4d24: Generated new TSK 0
I20260808 22:32:08.698359 11763 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.31.87.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.87.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.87.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.129.keytab.
Entry for principal oryx/127.31.87.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.129.keytab.
I20260808 22:32:08.731709 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.129:0
--local_ip_for_outbound_sockets=127.31.87.129
--webserver_interface=127.31.87.129
--webserver_port=0
--tserver_master_addrs=127.31.87.190:45745
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.129.keytab
--principal=oryx/127.31.87.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:38895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:32:08.839756 11791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:32:08.840001 11791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:32:08.840059 11791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:32:08.843933 11791 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:32:08.843995 11791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:32:08.844053 11791 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 22:32:08.844108 11791 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.129
I20260808 22:32:08.849134 11791 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:38895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.129.keytab
--principal=oryx/127.31.87.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.87.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.87.190:45745
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11791
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.129
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:32:08.850106 11791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:32:08.851066 11791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:32:08.860008 11805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:08.859998 11802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:08.860001 11799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:08.860262 11791 server_base.cc:1034] running on GCE node
I20260808 22:32:08.860631 11791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:32:08.861248 11791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:32:08.862401 11791 hybrid_clock.cc:648] HybridClock initialized: now 1786228328862379 us; error 43 us; skew 500 ppm
Aug 08 22:32:08 dist-test-slave-266d krb5kdc[11600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228328, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.87.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:32:08.864993 11791 init.cc:377] Logged in from keytab as oryx/127.31.87.129@KRBTEST.COM (short username oryx)
I20260808 22:32:08.865783 11791 webserver.cc:460] Webserver started at http://127.31.87.129:35913/ using document root <none> and password file <none>
I20260808 22:32:08.866286 11791 fs_manager.cc:359] Metadata directory not provided
I20260808 22:32:08.866349 11791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:32:08.866529 11791 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:32:08.868422 11791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-0/data/instance:
uuid: "c2b3097001874d8ab050b0b6f8823a76"
format_stamp: "Formatted at 2026-08-08 22:32:08 on dist-test-slave-266d"
server_key: "cf97285886bf2363d1be4131a366a949"
server_key_iv: "db2d41cc29e7c1ce4d1bc590c20cab74"
server_key_version: "encryptionkey@0"
I20260808 22:32:08.868928 11791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-0/wal/instance:
uuid: "c2b3097001874d8ab050b0b6f8823a76"
format_stamp: "Formatted at 2026-08-08 22:32:08 on dist-test-slave-266d"
server_key: "cf97285886bf2363d1be4131a366a949"
server_key_iv: "db2d41cc29e7c1ce4d1bc590c20cab74"
server_key_version: "encryptionkey@0"
I20260808 22:32:08.872299 11791 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 22:32:08.874524 11814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:08.875599 11791 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:32:08.875702 11791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-0/wal
uuid: "c2b3097001874d8ab050b0b6f8823a76"
format_stamp: "Formatted at 2026-08-08 22:32:08 on dist-test-slave-266d"
server_key: "cf97285886bf2363d1be4131a366a949"
server_key_iv: "db2d41cc29e7c1ce4d1bc590c20cab74"
server_key_version: "encryptionkey@0"
I20260808 22:32:08.875795 11791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:32:08.887012 11791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:32:08.889665 11791 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:32:08.889837 11791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:32:08.891188 11791 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:32:08.891237 11791 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:08.891288 11791 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:32:08.891317 11791 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 22:32:08 dist-test-slave-266d krb5kdc[11600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228328, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.87.129@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM
I20260808 22:32:08.900879 11791 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.129:40109
I20260808 22:32:08.900988 11999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.129:40109 every 8 connection(s)
I20260808 22:32:08.901870 11791 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
I20260808 22:32:08.903908 11748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:08.891882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:45969 (local address 127.31.87.190:45745)
0808 22:32:08.892020 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:08.892026 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:08.892846 (+ 820us) server_negotiation.cc:407] Connection header received
0808 22:32:08.893898 (+ 1052us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:08.893902 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:08.893962 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:08.894015 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:08.895661 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:08.896481 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:08.897842 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:08.898162 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:08.898254 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:08.900629 (+ 2375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:08.900633 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:08.900635 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:08.900664 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:08.902141 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:08.902684 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:08.902688 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:08.902690 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:08.902771 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:08.903050 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:08.903052 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:08.903054 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:08.903209 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:08.903289 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:08.903692 (+ 403us) server_negotiation.cc:299] Negotiation successful
0808 22:32:08.903789 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 22:32:08.904621 11831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:08.892248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45745 (local address 127.31.87.129:45969)
0808 22:32:08.892682 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:08.892730 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:08.893655 (+ 925us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:08.894424 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:08.894433 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:08.894863 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:08.895483 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:08.895503 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:08.896598 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:08.896601 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:08.897735 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:08.897745 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:08.898309 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:08.898314 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:08.898487 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:08.899019 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:08.899040 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:08.900470 (+ 1430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:32:08.902244 (+ 1774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:08.902250 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:08.902265 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:08.902592 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:08.902855 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:08.902857 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:08.902859 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:08.902983 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:08.903279 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:32:08.903283 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:32:08.903488 (+ 205us) client_negotiation.cc:769] Sending connection context
0808 22:32:08.903662 (+ 174us) client_negotiation.cc:240] Negotiation successful
0808 22:32:08.903888 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":98,"threads_started":1}
I20260808 22:32:08.905710 12000 heartbeater.cc:347] Connected to a master server at 127.31.87.190:45745
I20260808 22:32:08.905849 12000 heartbeater.cc:464] Registering TS with master...
I20260808 22:32:08.906291 12000 heartbeater.cc:511] Master 127.31.87.190:45745 requested a full tablet report, sending...
I20260808 22:32:08.907728 11671 ts_manager.cc:194] Registered new tserver with Master: c2b3097001874d8ab050b0b6f8823a76 (127.31.87.129:40109)
I20260808 22:32:08.908318 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 11791
I20260808 22:32:08.908453 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-0/wal/instance
I20260808 22:32:08.908808 32094 external_mini_cluster.cc:1428] Setting key e5bd0272ac950949fb946b1b894c8363
I20260808 22:32:08.909109 11671 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.87.129@KRBTEST.COM'} at 127.31.87.129:45969
I20260808 22:32:08.915035 12008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:08.907726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45745 (local address 127.31.87.129:48635)
0808 22:32:08.907984 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:08.907995 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:08.908081 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:08.908355 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:08.908358 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:08.908548 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:08.908684 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:08.908690 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:08.909624 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:08.909628 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:08.910410 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:08.910421 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:08.910911 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:08.910915 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:08.911002 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:08.911495 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:08.911509 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:08.911810 (+ 301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:32:08.913295 (+ 1485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:08.913316 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:08.913318 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:08.913551 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:08.913864 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:08.913867 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:08.913869 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:08.913964 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:08.914459 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:32:08.914463 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:32:08.914541 (+ 78us) client_negotiation.cc:769] Sending connection context
0808 22:32:08.914690 (+ 149us) client_negotiation.cc:240] Negotiation successful
0808 22:32:08.914832 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":4,"thread_start_us":101,"threads_started":1}
I20260808 22:32:08.915350 11748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:08.907794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:48635 (local address 127.31.87.190:45745)
0808 22:32:08.907896 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:08.907900 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:08.908056 (+ 156us) server_negotiation.cc:407] Connection header received
0808 22:32:08.908197 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:08.908200 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:08.908241 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:08.908326 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:08.908806 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:08.909498 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:08.910537 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:08.910791 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:08.910891 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:08.911935 (+ 1044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:08.911939 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:08.911941 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:08.911965 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:08.913199 (+ 1234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:08.913649 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:08.913652 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:08.913654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:08.913762 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:08.914065 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:08.914068 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:08.914070 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:08.914315 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:08.914413 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:08.914849 (+ 436us) server_negotiation.cc:299] Negotiation successful
0808 22:32:08.915020 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":35}
WARNING: no policy specified for oryx/127.31.87.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.87.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.87.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.130.keytab.
Entry for principal oryx/127.31.87.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.130.keytab.
I20260808 22:32:08.935458 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.130:0
--local_ip_for_outbound_sockets=127.31.87.130
--webserver_interface=127.31.87.130
--webserver_port=0
--tserver_master_addrs=127.31.87.190:45745
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.130.keytab
--principal=oryx/127.31.87.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:38895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:32:09.043345 12013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:32:09.043596 12013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:32:09.043668 12013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:32:09.047564 12013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:32:09.047622 12013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:32:09.047676 12013 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 22:32:09.047731 12013 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.130
I20260808 22:32:09.052702 12013 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:38895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.130.keytab
--principal=oryx/127.31.87.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.87.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.87.190:45745
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12013
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.130
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:32:09.053675 12013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:32:09.054636 12013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:32:09.063818 12022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:09.063820 12024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:09.064072 12021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:09.063907 12013 server_base.cc:1034] running on GCE node
I20260808 22:32:09.064479 12013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:32:09.065047 12013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:32:09.066232 12013 hybrid_clock.cc:648] HybridClock initialized: now 1786228329066214 us; error 28 us; skew 500 ppm
Aug 08 22:32:09 dist-test-slave-266d krb5kdc[11600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228329, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.87.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:32:09.068715 12013 init.cc:377] Logged in from keytab as oryx/127.31.87.130@KRBTEST.COM (short username oryx)
I20260808 22:32:09.069563 12013 webserver.cc:460] Webserver started at http://127.31.87.130:34141/ using document root <none> and password file <none>
I20260808 22:32:09.070075 12013 fs_manager.cc:359] Metadata directory not provided
I20260808 22:32:09.070149 12013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:32:09.070348 12013 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:32:09.072247 12013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-1/data/instance:
uuid: "35c0c3409b0145eba8283e58d3080673"
format_stamp: "Formatted at 2026-08-08 22:32:09 on dist-test-slave-266d"
server_key: "1ef8bc5505fbacc943fa352568e9b795"
server_key_iv: "c51971c18e2f7891b692c25183688f36"
server_key_version: "encryptionkey@0"
I20260808 22:32:09.072770 12013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-1/wal/instance:
uuid: "35c0c3409b0145eba8283e58d3080673"
format_stamp: "Formatted at 2026-08-08 22:32:09 on dist-test-slave-266d"
server_key: "1ef8bc5505fbacc943fa352568e9b795"
server_key_iv: "c51971c18e2f7891b692c25183688f36"
server_key_version: "encryptionkey@0"
I20260808 22:32:09.076229 12013 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 22:32:09.078611 12036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:09.079828 12013 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:32:09.079919 12013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-1/wal
uuid: "35c0c3409b0145eba8283e58d3080673"
format_stamp: "Formatted at 2026-08-08 22:32:09 on dist-test-slave-266d"
server_key: "1ef8bc5505fbacc943fa352568e9b795"
server_key_iv: "c51971c18e2f7891b692c25183688f36"
server_key_version: "encryptionkey@0"
I20260808 22:32:09.079999 12013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:32:09.089862 12013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:32:09.092373 12013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:32:09.092515 12013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:32:09.093959 12013 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:32:09.094012 12013 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:09.094048 12013 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:32:09.094089 12013 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 22:32:09 dist-test-slave-266d krb5kdc[11600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228329, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.87.130@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM
I20260808 22:32:09.103811 12013 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.130:41999
I20260808 22:32:09.103869 12229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.130:41999 every 8 connection(s)
I20260808 22:32:09.104664 12013 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
I20260808 22:32:09.107196 11748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:09.094593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:44401 (local address 127.31.87.190:45745)
0808 22:32:09.094706 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:09.094710 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:09.095703 (+ 993us) server_negotiation.cc:407] Connection header received
0808 22:32:09.096677 (+ 974us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:09.096682 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:09.096741 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:09.096813 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:09.098508 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.099314 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.100937 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.101283 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.101402 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:09.103760 (+ 2358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:09.103773 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:09.103776 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:09.103806 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:09.105345 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:09.105921 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:09.105924 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:09.105926 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:09.106004 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:09.106310 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:09.106313 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:09.106315 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:09.106478 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:09.106551 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:09.106972 (+ 421us) server_negotiation.cc:299] Negotiation successful
0808 22:32:09.107074 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 22:32:09.107957 12053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:09.095101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45745 (local address 127.31.87.130:44401)
0808 22:32:09.095562 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:09.095602 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:09.096445 (+ 843us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:09.097259 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:09.097267 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:09.097696 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:09.098316 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:09.098337 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.099446 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.099450 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:09.100805 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:09.100817 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.101425 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.101428 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:09.101561 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:09.102092 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:09.102112 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:09.103572 (+ 1460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:32:09.105455 (+ 1883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:09.105461 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:09.105474 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:09.105821 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:09.106073 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:09.106075 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:09.106077 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:09.106239 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:09.106550 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:32:09.106554 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:32:09.106762 (+ 208us) client_negotiation.cc:769] Sending connection context
0808 22:32:09.106918 (+ 156us) client_negotiation.cc:240] Negotiation successful
0808 22:32:09.107180 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":122,"threads_started":1}
I20260808 22:32:09.109128 12234 heartbeater.cc:347] Connected to a master server at 127.31.87.190:45745
I20260808 22:32:09.109272 12234 heartbeater.cc:464] Registering TS with master...
I20260808 22:32:09.109781 12234 heartbeater.cc:511] Master 127.31.87.190:45745 requested a full tablet report, sending...
I20260808 22:32:09.110828 11671 ts_manager.cc:194] Registered new tserver with Master: 35c0c3409b0145eba8283e58d3080673 (127.31.87.130:41999)
I20260808 22:32:09.110986 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 12013
I20260808 22:32:09.111104 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-1/wal/instance
I20260808 22:32:09.111425 32094 external_mini_cluster.cc:1428] Setting key 34d2967f2fd186e369d01f0f42c39dbf
I20260808 22:32:09.111548 11671 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.87.130@KRBTEST.COM'} at 127.31.87.130:44401
I20260808 22:32:09.118705 11748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:09.111285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:45571 (local address 127.31.87.190:45745)
0808 22:32:09.111392 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:09.111396 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:09.111986 (+ 590us) server_negotiation.cc:407] Connection header received
0808 22:32:09.112186 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:09.112189 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:09.112232 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:09.112325 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:09.112691 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.113440 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.114489 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.114754 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.114852 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:09.115893 (+ 1041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:09.115897 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:09.115900 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:09.115925 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:09.117256 (+ 1331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:09.117616 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:09.117619 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:09.117621 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:09.117690 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:09.117941 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:09.117944 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:09.117946 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:09.118134 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:09.118219 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:09.118464 (+ 245us) server_negotiation.cc:299] Negotiation successful
0808 22:32:09.118558 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 22:32:09.118717 12246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:09.111229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45745 (local address 127.31.87.130:45571)
0808 22:32:09.111914 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:09.111926 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:09.112051 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:09.112336 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:09.112338 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:09.112485 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:09.112581 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:09.112588 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.113576 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.113579 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:09.114358 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:09.114369 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.114962 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.114965 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:09.115037 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:09.115451 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:09.115462 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:09.115748 (+ 286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:32:09.117345 (+ 1597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:09.117349 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:09.117351 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:09.117535 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:09.117767 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:09.117771 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:09.117772 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:09.117865 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:09.118222 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:32:09.118225 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:32:09.118317 (+ 92us) client_negotiation.cc:769] Sending connection context
0808 22:32:09.118400 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 22:32:09.118529 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":106,"threads_started":1}
WARNING: no policy specified for oryx/127.31.87.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.87.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.87.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.131.keytab.
Entry for principal oryx/127.31.87.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.131.keytab.
I20260808 22:32:09.137176 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.131:0
--local_ip_for_outbound_sockets=127.31.87.131
--webserver_interface=127.31.87.131
--webserver_port=0
--tserver_master_addrs=127.31.87.190:45745
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.131.keytab
--principal=oryx/127.31.87.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:38895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:32:09.245854 12251 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:32:09.246084 12251 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:32:09.246133 12251 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:32:09.249969 12251 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:32:09.250020 12251 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:32:09.250056 12251 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 22:32:09.250092 12251 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.131
I20260808 22:32:09.255095 12251 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:38895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.131.keytab
--principal=oryx/127.31.87.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.87.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.87.190:45745
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12251
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.131
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:32:09.255980 12251 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:32:09.256898 12251 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:32:09.266095 12263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:09.266064 12262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:09.266070 12265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:09.266165 12251 server_base.cc:1034] running on GCE node
I20260808 22:32:09.266724 12251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:32:09.267275 12251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:32:09.268420 12251 hybrid_clock.cc:648] HybridClock initialized: now 1786228329268396 us; error 37 us; skew 500 ppm
Aug 08 22:32:09 dist-test-slave-266d krb5kdc[11600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228329, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.87.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:32:09.271041 12251 init.cc:377] Logged in from keytab as oryx/127.31.87.131@KRBTEST.COM (short username oryx)
I20260808 22:32:09.271844 12251 webserver.cc:460] Webserver started at http://127.31.87.131:45439/ using document root <none> and password file <none>
I20260808 22:32:09.272367 12251 fs_manager.cc:359] Metadata directory not provided
I20260808 22:32:09.272430 12251 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:32:09.272626 12251 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:32:09.274636 12251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-2/data/instance:
uuid: "a8382975b0ef4c78a8aa87bb78abda54"
format_stamp: "Formatted at 2026-08-08 22:32:09 on dist-test-slave-266d"
server_key: "61bf9084b976a5bdb02645036c7a106f"
server_key_iv: "8005663f7aba8fed607d251623b925a0"
server_key_version: "encryptionkey@0"
I20260808 22:32:09.275141 12251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-2/wal/instance:
uuid: "a8382975b0ef4c78a8aa87bb78abda54"
format_stamp: "Formatted at 2026-08-08 22:32:09 on dist-test-slave-266d"
server_key: "61bf9084b976a5bdb02645036c7a106f"
server_key_iv: "8005663f7aba8fed607d251623b925a0"
server_key_version: "encryptionkey@0"
I20260808 22:32:09.278627 12251 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 22:32:09.280910 12280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:09.282049 12251 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:32:09.282152 12251 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-2/wal
uuid: "a8382975b0ef4c78a8aa87bb78abda54"
format_stamp: "Formatted at 2026-08-08 22:32:09 on dist-test-slave-266d"
server_key: "61bf9084b976a5bdb02645036c7a106f"
server_key_iv: "8005663f7aba8fed607d251623b925a0"
server_key_version: "encryptionkey@0"
I20260808 22:32:09.282239 12251 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:32:09.304594 12251 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:32:09.307670 12251 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:32:09.307840 12251 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:32:09.309590 12251 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:32:09.309652 12251 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:09.309711 12251 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:32:09.309742 12251 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 22:32:09 dist-test-slave-266d krb5kdc[11600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228329, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.87.131@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM
I20260808 22:32:09.321485 12463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.131:36531 every 8 connection(s)
I20260808 22:32:09.321482 12251 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.131:36531
I20260808 22:32:09.322669 12251 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
I20260808 22:32:09.323320 11748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:09.310075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:34665 (local address 127.31.87.190:45745)
0808 22:32:09.310229 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:09.310234 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:09.311207 (+ 973us) server_negotiation.cc:407] Connection header received
0808 22:32:09.312445 (+ 1238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:09.312448 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:09.312509 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:09.312635 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:09.314242 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.314883 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.316116 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.316352 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.316442 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:09.319322 (+ 2880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:09.319326 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:09.319329 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:09.319356 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:09.321015 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:09.321700 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:09.321703 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:09.321704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:09.321781 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:09.322150 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:09.322153 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:09.322154 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:09.322326 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:09.322399 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:09.323017 (+ 618us) server_negotiation.cc:299] Negotiation successful
0808 22:32:09.323164 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 22:32:09.323693 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 12251
I20260808 22:32:09.323820 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-2/wal/instance
I20260808 22:32:09.324115 32094 external_mini_cluster.cc:1428] Setting key 4b95baae935c8f979a0c6f2946503a45
I20260808 22:32:09.324321 12295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:09.310520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45745 (local address 127.31.87.131:34665)
0808 22:32:09.311010 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:09.311082 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:09.312176 (+ 1094us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:09.312824 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:09.312836 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:09.313352 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:09.314042 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:09.314076 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.315019 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.315022 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:09.316000 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:09.316011 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.316492 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.316496 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:09.316659 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:09.317214 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:09.317235 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:09.319166 (+ 1931us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:32:09.321211 (+ 2045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:09.321217 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:09.321233 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:09.321589 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:09.321863 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:09.321866 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:09.321868 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:09.322034 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:09.322425 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:32:09.322432 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:32:09.322765 (+ 333us) client_negotiation.cc:769] Sending connection context
0808 22:32:09.322986 (+ 221us) client_negotiation.cc:240] Negotiation successful
0808 22:32:09.323404 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":105,"threads_started":1}
I20260808 22:32:09.325570 12469 heartbeater.cc:347] Connected to a master server at 127.31.87.190:45745
I20260808 22:32:09.325763 12469 heartbeater.cc:464] Registering TS with master...
I20260808 22:32:09.326330 12469 heartbeater.cc:511] Master 127.31.87.190:45745 requested a full tablet report, sending...
I20260808 22:32:09.327322 11672 ts_manager.cc:194] Registered new tserver with Master: a8382975b0ef4c78a8aa87bb78abda54 (127.31.87.131:36531)
I20260808 22:32:09.327955 11672 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.87.131@KRBTEST.COM'} at 127.31.87.131:34665
I20260808 22:32:09.333499 11748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:09.327536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:51275 (local address 127.31.87.190:45745)
0808 22:32:09.327675 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:09.327679 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:09.327846 (+ 167us) server_negotiation.cc:407] Connection header received
0808 22:32:09.327936 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:09.327939 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:09.327978 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:09.328039 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:09.328483 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.329277 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.330059 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.330268 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.330355 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:09.331247 (+ 892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:09.331250 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:09.331252 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:09.331275 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:09.332296 (+ 1021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:09.332638 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:09.332641 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:09.332642 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:09.332700 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:09.332882 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:09.332883 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:09.332884 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:09.333021 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:09.333119 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:09.333317 (+ 198us) server_negotiation.cc:299] Negotiation successful
0808 22:32:09.333397 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 22:32:09.333509 12476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:09.327485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45745 (local address 127.31.87.131:51275)
0808 22:32:09.327772 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:09.327784 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:09.327868 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:09.328064 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:09.328067 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:09.328246 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:09.328366 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:09.328373 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.329390 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.329393 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:09.329940 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:09.329948 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.330365 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.330368 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:09.330436 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:09.330830 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:09.330840 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:09.331131 (+ 291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:32:09.332386 (+ 1255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:09.332389 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:09.332390 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:09.332569 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:09.332759 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:09.332761 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:09.332762 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:09.332825 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:09.333081 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:32:09.333083 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:32:09.333198 (+ 115us) client_negotiation.cc:769] Sending connection context
0808 22:32:09.333277 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 22:32:09.333372 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":75,"threads_started":1}
I20260808 22:32:09.338713 32094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 22:32:09 dist-test-slave-266d krb5kdc[11600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228329, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 22:32:09 dist-test-slave-266d krb5kdc[11600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:32:09 dist-test-slave-266d krb5kdc[11600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 22:32:09.361931 11748 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:09.356353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33818 (local address 127.31.87.190:45745)
0808 22:32:09.356480 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:09.356483 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:09.356493 (+ 10us) server_negotiation.cc:407] Connection header received
0808 22:32:09.356528 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:09.356530 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:09.356566 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:09.356654 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:09.357356 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.357867 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.358683 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.358893 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.358978 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:09.361814 (+ 2836us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33818: BlockingRecv error: recv got EOF from 127.0.0.1:33818 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 08 22:32:09 dist-test-slave-266d krb5kdc[11600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228329, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM
I20260808 22:32:09.371476 11748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:09.364452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33820 (local address 127.31.87.190:45745)
0808 22:32:09.364549 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:09.364553 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:09.364584 (+ 31us) server_negotiation.cc:407] Connection header received
0808 22:32:09.364617 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:09.364619 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:09.364656 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:09.364725 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:09.365293 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.365781 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.366480 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.366701 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.366784 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:09.368869 (+ 2085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:09.368872 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:09.368874 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:09.368897 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:09.370002 (+ 1105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:09.370387 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:09.370391 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:09.370392 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:09.370463 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:09.370709 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:09.370712 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:09.370714 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:09.370880 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:09.370995 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:09.371230 (+ 235us) server_negotiation.cc:299] Negotiation successful
0808 22:32:09.371332 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 22:32:09.374490 11672 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33820:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:32:09.377453 11672 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:32:09.390836 12511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:09.385399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.130:41999 (local address 127.0.0.1:34662)
0808 22:32:09.385895 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:09.385940 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:09.386059 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:09.386805 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:09.386811 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:09.386836 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:32:09.386963 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:09.386970 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.388115 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.388119 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:09.389258 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:09.389277 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.390338 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.390342 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:09.390449 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:09.390489 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 22:32:09.390585 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 22:32:09.390696 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":80,"threads_started":1}
I20260808 22:32:09.391014 12514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:09.385545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:40109 (local address 127.0.0.1:57510)
0808 22:32:09.386298 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:09.386310 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:09.386390 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:09.387024 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:09.387027 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:09.387045 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:32:09.387155 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:09.387162 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.388631 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.388634 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:09.389702 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:09.389712 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.390634 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.390638 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:09.390728 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:09.390752 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 22:32:09.390819 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 22:32:09.390884 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":696,"thread_start_us":101,"threads_started":1}
I20260808 22:32:09.391072 12517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:09.386254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57510 (local address 127.31.87.129:40109)
0808 22:32:09.386590 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:09.386606 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:09.386627 (+ 21us) server_negotiation.cc:407] Connection header received
0808 22:32:09.386697 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:09.386702 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:09.386877 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:09.387297 (+ 420us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:32:09.387427 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.388506 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.389843 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.390489 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.390594 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:09.390741 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:09.390846 (+ 105us) server_negotiation.cc:299] Negotiation successful
0808 22:32:09.390937 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":82,"threads_started":1}
I20260808 22:32:09.391577 12509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:09.386095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.131:36531 (local address 127.0.0.1:38168)
0808 22:32:09.387025 (+ 930us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:09.387037 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:09.387106 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:09.387590 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:09.387593 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:09.387612 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:32:09.387694 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:09.387699 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.388871 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.388874 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:09.389846 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:09.389857 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.391215 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.391219 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:09.391304 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:09.391327 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 22:32:09.391399 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 22:32:09.391461 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":871,"thread_start_us":89,"threads_started":1}
I20260808 22:32:09.391597 12512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:09.385558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34662 (local address 127.31.87.130:41999)
0808 22:32:09.386270 (+ 712us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:09.386281 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:09.386306 (+ 25us) server_negotiation.cc:407] Connection header received
0808 22:32:09.386384 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:09.386389 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:09.386589 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:09.386792 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:32:09.387093 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.388027 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.389407 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.390218 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.390320 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:09.390472 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:09.391380 (+ 908us) server_negotiation.cc:299] Negotiation successful
0808 22:32:09.391471 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":105,"threads_started":1}
I20260808 22:32:09.391690 12518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:09.386403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38168 (local address 127.31.87.131:36531)
0808 22:32:09.386702 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:09.386714 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:09.387194 (+ 480us) server_negotiation.cc:407] Connection header received
0808 22:32:09.387271 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:09.387276 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:09.387450 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:09.387597 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:32:09.387802 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.388775 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.389984 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.390738 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.390842 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:09.390987 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:09.391450 (+ 463us) server_negotiation.cc:299] Negotiation successful
0808 22:32:09.391547 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":109,"threads_started":1}
I20260808 22:32:09.394057 12121 tablet_service.cc:1467] Processing CreateTablet for tablet 096d509446e94e628cd32452bd06565a (DEFAULT_TABLE table=test-table [id=35578e809d084038b4a8258fbeb791da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:32:09.394101 11900 tablet_service.cc:1467] Processing CreateTablet for tablet 096d509446e94e628cd32452bd06565a (DEFAULT_TABLE table=test-table [id=35578e809d084038b4a8258fbeb791da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:32:09.395023 12121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 096d509446e94e628cd32452bd06565a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:32:09.394717 12360 tablet_service.cc:1467] Processing CreateTablet for tablet 096d509446e94e628cd32452bd06565a (DEFAULT_TABLE table=test-table [id=35578e809d084038b4a8258fbeb791da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:32:09.395098 11900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 096d509446e94e628cd32452bd06565a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:32:09.395597 12360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 096d509446e94e628cd32452bd06565a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:32:09.402223 12519 tablet_bootstrap.cc:492] T 096d509446e94e628cd32452bd06565a P 35c0c3409b0145eba8283e58d3080673: Bootstrap starting.
I20260808 22:32:09.403031 12523 tablet_bootstrap.cc:492] T 096d509446e94e628cd32452bd06565a P a8382975b0ef4c78a8aa87bb78abda54: Bootstrap starting.
I20260808 22:32:09.404285 12519 tablet_bootstrap.cc:654] T 096d509446e94e628cd32452bd06565a P 35c0c3409b0145eba8283e58d3080673: Neither blocks nor log segments found. Creating new log.
I20260808 22:32:09.404681 12523 tablet_bootstrap.cc:654] T 096d509446e94e628cd32452bd06565a P a8382975b0ef4c78a8aa87bb78abda54: Neither blocks nor log segments found. Creating new log.
I20260808 22:32:09.404982 12519 log.cc:824] T 096d509446e94e628cd32452bd06565a P 35c0c3409b0145eba8283e58d3080673: Log is configured to *not* fsync() on all Append() calls
I20260808 22:32:09.405462 12523 log.cc:824] T 096d509446e94e628cd32452bd06565a P a8382975b0ef4c78a8aa87bb78abda54: Log is configured to *not* fsync() on all Append() calls
I20260808 22:32:09.405660 12521 tablet_bootstrap.cc:492] T 096d509446e94e628cd32452bd06565a P c2b3097001874d8ab050b0b6f8823a76: Bootstrap starting.
I20260808 22:32:09.407068 12519 tablet_bootstrap.cc:492] T 096d509446e94e628cd32452bd06565a P 35c0c3409b0145eba8283e58d3080673: No bootstrap required, opened a new log
I20260808 22:32:09.407258 12519 ts_tablet_manager.cc:1397] T 096d509446e94e628cd32452bd06565a P 35c0c3409b0145eba8283e58d3080673: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 22:32:09.407356 12523 tablet_bootstrap.cc:492] T 096d509446e94e628cd32452bd06565a P a8382975b0ef4c78a8aa87bb78abda54: No bootstrap required, opened a new log
I20260808 22:32:09.407526 12523 ts_tablet_manager.cc:1397] T 096d509446e94e628cd32452bd06565a P a8382975b0ef4c78a8aa87bb78abda54: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 22:32:09.407925 12521 tablet_bootstrap.cc:654] T 096d509446e94e628cd32452bd06565a P c2b3097001874d8ab050b0b6f8823a76: Neither blocks nor log segments found. Creating new log.
I20260808 22:32:09.408530 12521 log.cc:824] T 096d509446e94e628cd32452bd06565a P c2b3097001874d8ab050b0b6f8823a76: Log is configured to *not* fsync() on all Append() calls
I20260808 22:32:09.410110 12521 tablet_bootstrap.cc:492] T 096d509446e94e628cd32452bd06565a P c2b3097001874d8ab050b0b6f8823a76: No bootstrap required, opened a new log
I20260808 22:32:09.410281 12521 ts_tablet_manager.cc:1397] T 096d509446e94e628cd32452bd06565a P c2b3097001874d8ab050b0b6f8823a76: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 22:32:09.411114 12519 raft_consensus.cc:348] T 096d509446e94e628cd32452bd06565a P 35c0c3409b0145eba8283e58d3080673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } } peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } }
I20260808 22:32:09.411123 12523 raft_consensus.cc:348] T 096d509446e94e628cd32452bd06565a P a8382975b0ef4c78a8aa87bb78abda54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } } peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } }
I20260808 22:32:09.411371 12519 raft_consensus.cc:374] T 096d509446e94e628cd32452bd06565a P 35c0c3409b0145eba8283e58d3080673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:32:09.411371 12523 raft_consensus.cc:374] T 096d509446e94e628cd32452bd06565a P a8382975b0ef4c78a8aa87bb78abda54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:32:09.411427 12523 raft_consensus.cc:729] T 096d509446e94e628cd32452bd06565a P a8382975b0ef4c78a8aa87bb78abda54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8382975b0ef4c78a8aa87bb78abda54, State: Initialized, Role: FOLLOWER
I20260808 22:32:09.411427 12519 raft_consensus.cc:729] T 096d509446e94e628cd32452bd06565a P 35c0c3409b0145eba8283e58d3080673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35c0c3409b0145eba8283e58d3080673, State: Initialized, Role: FOLLOWER
I20260808 22:32:09.411865 12523 consensus_queue.cc:260] T 096d509446e94e628cd32452bd06565a P a8382975b0ef4c78a8aa87bb78abda54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } } peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } }
I20260808 22:32:09.411865 12519 consensus_queue.cc:260] T 096d509446e94e628cd32452bd06565a P 35c0c3409b0145eba8283e58d3080673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } } peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } }
I20260808 22:32:09.412497 12234 heartbeater.cc:503] Master 127.31.87.190:45745 was elected leader, sending a full tablet report...
I20260808 22:32:09.412835 12523 ts_tablet_manager.cc:1428] T 096d509446e94e628cd32452bd06565a P a8382975b0ef4c78a8aa87bb78abda54: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 22:32:09.412835 12519 ts_tablet_manager.cc:1428] T 096d509446e94e628cd32452bd06565a P 35c0c3409b0145eba8283e58d3080673: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 22:32:09.413175 12469 heartbeater.cc:503] Master 127.31.87.190:45745 was elected leader, sending a full tablet report...
I20260808 22:32:09.414208 12521 raft_consensus.cc:348] T 096d509446e94e628cd32452bd06565a P c2b3097001874d8ab050b0b6f8823a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } } peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } }
I20260808 22:32:09.414443 12521 raft_consensus.cc:374] T 096d509446e94e628cd32452bd06565a P c2b3097001874d8ab050b0b6f8823a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:32:09.414505 12521 raft_consensus.cc:729] T 096d509446e94e628cd32452bd06565a P c2b3097001874d8ab050b0b6f8823a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2b3097001874d8ab050b0b6f8823a76, State: Initialized, Role: FOLLOWER
I20260808 22:32:09.414896 12521 consensus_queue.cc:260] T 096d509446e94e628cd32452bd06565a P c2b3097001874d8ab050b0b6f8823a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } } peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } }
I20260808 22:32:09.415498 12000 heartbeater.cc:503] Master 127.31.87.190:45745 was elected leader, sending a full tablet report...
I20260808 22:32:09.415853 12521 ts_tablet_manager.cc:1428] T 096d509446e94e628cd32452bd06565a P c2b3097001874d8ab050b0b6f8823a76: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 22:32:09.556808 12532 raft_consensus.cc:482] T 096d509446e94e628cd32452bd06565a P a8382975b0ef4c78a8aa87bb78abda54 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:32:09.557044 12532 raft_consensus.cc:504] T 096d509446e94e628cd32452bd06565a P a8382975b0ef4c78a8aa87bb78abda54 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } } peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } }
I20260808 22:32:09.558320 12532 leader_election.cc:288] T 096d509446e94e628cd32452bd06565a P a8382975b0ef4c78a8aa87bb78abda54 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35c0c3409b0145eba8283e58d3080673 (127.31.87.130:41999), c2b3097001874d8ab050b0b6f8823a76 (127.31.87.129:40109)
I20260808 22:32:09.562016 12295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:09.558587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:40109 (local address 127.31.87.131:57951)
0808 22:32:09.558707 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:09.558722 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:09.558820 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:09.559088 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:09.559091 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:09.559112 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:32:09.559233 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:09.559241 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.560025 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.560028 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:09.561000 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:09.561011 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.561650 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.561652 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:09.561732 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:09.561753 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 22:32:09.561807 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 22:32:09.561903 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 22:32:09.562049 12517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:09.558656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:57951 (local address 127.31.87.129:40109)
0808 22:32:09.558778 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:09.558783 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:09.558802 (+ 19us) server_negotiation.cc:407] Connection header received
0808 22:32:09.558930 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:09.558933 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:09.558988 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:09.559056 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:32:09.559343 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.559919 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.561155 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.561543 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.561659 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:09.561715 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:09.561851 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 22:32:09.561920 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 22:32:09.562680 12512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:09.558672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:58401 (local address 127.31.87.130:41999)
0808 22:32:09.558798 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:09.558802 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:09.559171 (+ 369us) server_negotiation.cc:407] Connection header received
0808 22:32:09.559282 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:09.559285 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:09.559332 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:09.559405 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:32:09.559928 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.560433 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.561584 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.562117 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.562225 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:09.562286 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:09.562512 (+ 226us) server_negotiation.cc:299] Negotiation successful
0808 22:32:09.562578 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 22:32:09.562690 12535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:09.558586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.130:41999 (local address 127.31.87.131:58401)
0808 22:32:09.559076 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:09.559089 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:09.559187 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:09.559419 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:09.559421 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:09.559435 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:32:09.559823 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:09.559829 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.560540 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.560543 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:09.561479 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:09.561488 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.562260 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.562264 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:09.562358 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:09.562379 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 22:32:09.562488 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 22:32:09.562553 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"mutex_wait_us":131,"thread_start_us":153,"threads_started":1}
I20260808 22:32:09.562731 11932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "096d509446e94e628cd32452bd06565a" candidate_uuid: "a8382975b0ef4c78a8aa87bb78abda54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2b3097001874d8ab050b0b6f8823a76" is_pre_election: true
I20260808 22:32:09.563073 11932 raft_consensus.cc:2454] T 096d509446e94e628cd32452bd06565a P c2b3097001874d8ab050b0b6f8823a76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a8382975b0ef4c78a8aa87bb78abda54 in term 0.
I20260808 22:32:09.563210 12151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "096d509446e94e628cd32452bd06565a" candidate_uuid: "a8382975b0ef4c78a8aa87bb78abda54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35c0c3409b0145eba8283e58d3080673" is_pre_election: true
I20260808 22:32:09.563480 12151 raft_consensus.cc:2454] T 096d509446e94e628cd32452bd06565a P 35c0c3409b0145eba8283e58d3080673 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a8382975b0ef4c78a8aa87bb78abda54 in term 0.
I20260808 22:32:09.563616 12285 leader_election.cc:302] T 096d509446e94e628cd32452bd06565a P a8382975b0ef4c78a8aa87bb78abda54 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8382975b0ef4c78a8aa87bb78abda54, c2b3097001874d8ab050b0b6f8823a76; no voters:
I20260808 22:32:09.563882 12532 raft_consensus.cc:2781] T 096d509446e94e628cd32452bd06565a P a8382975b0ef4c78a8aa87bb78abda54 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:32:09.563972 12532 raft_consensus.cc:482] T 096d509446e94e628cd32452bd06565a P a8382975b0ef4c78a8aa87bb78abda54 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:32:09.564021 12532 raft_consensus.cc:3037] T 096d509446e94e628cd32452bd06565a P a8382975b0ef4c78a8aa87bb78abda54 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:32:09.565011 12532 raft_consensus.cc:504] T 096d509446e94e628cd32452bd06565a P a8382975b0ef4c78a8aa87bb78abda54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } } peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } }
I20260808 22:32:09.565337 12532 leader_election.cc:288] T 096d509446e94e628cd32452bd06565a P a8382975b0ef4c78a8aa87bb78abda54 [CANDIDATE]: Term 1 election: Requested vote from peers 35c0c3409b0145eba8283e58d3080673 (127.31.87.130:41999), c2b3097001874d8ab050b0b6f8823a76 (127.31.87.129:40109)
I20260808 22:32:09.565654 11932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "096d509446e94e628cd32452bd06565a" candidate_uuid: "a8382975b0ef4c78a8aa87bb78abda54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2b3097001874d8ab050b0b6f8823a76"
I20260808 22:32:09.565654 12151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "096d509446e94e628cd32452bd06565a" candidate_uuid: "a8382975b0ef4c78a8aa87bb78abda54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35c0c3409b0145eba8283e58d3080673"
I20260808 22:32:09.565771 12151 raft_consensus.cc:3037] T 096d509446e94e628cd32452bd06565a P 35c0c3409b0145eba8283e58d3080673 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:32:09.565771 11932 raft_consensus.cc:3037] T 096d509446e94e628cd32452bd06565a P c2b3097001874d8ab050b0b6f8823a76 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:32:09.566967 12151 raft_consensus.cc:2454] T 096d509446e94e628cd32452bd06565a P 35c0c3409b0145eba8283e58d3080673 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8382975b0ef4c78a8aa87bb78abda54 in term 1.
I20260808 22:32:09.566967 11932 raft_consensus.cc:2454] T 096d509446e94e628cd32452bd06565a P c2b3097001874d8ab050b0b6f8823a76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8382975b0ef4c78a8aa87bb78abda54 in term 1.
I20260808 22:32:09.567256 12281 leader_election.cc:302] T 096d509446e94e628cd32452bd06565a P a8382975b0ef4c78a8aa87bb78abda54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35c0c3409b0145eba8283e58d3080673, a8382975b0ef4c78a8aa87bb78abda54; no voters:
I20260808 22:32:09.567410 12532 raft_consensus.cc:2781] T 096d509446e94e628cd32452bd06565a P a8382975b0ef4c78a8aa87bb78abda54 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:32:09.567615 12532 raft_consensus.cc:686] T 096d509446e94e628cd32452bd06565a P a8382975b0ef4c78a8aa87bb78abda54 [term 1 LEADER]: Becoming Leader. State: Replica: a8382975b0ef4c78a8aa87bb78abda54, State: Running, Role: LEADER
I20260808 22:32:09.567938 12532 consensus_queue.cc:237] T 096d509446e94e628cd32452bd06565a P a8382975b0ef4c78a8aa87bb78abda54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } } peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } }
I20260808 22:32:09.571182 11672 catalog_manager.cc:5495] T 096d509446e94e628cd32452bd06565a P a8382975b0ef4c78a8aa87bb78abda54 reported cstate change: term changed from 0 to 1, leader changed from <none> to a8382975b0ef4c78a8aa87bb78abda54 (127.31.87.131). New cstate: current_term: 1 leader_uuid: "a8382975b0ef4c78a8aa87bb78abda54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } health_report { overall_health: HEALTHY } } }
W20260808 22:32:09.573936 12470 tablet.cc:2369] T 096d509446e94e628cd32452bd06565a P a8382975b0ef4c78a8aa87bb78abda54: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 22:32:09.605710 12236 tablet.cc:2369] T 096d509446e94e628cd32452bd06565a P 35c0c3409b0145eba8283e58d3080673: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:32:09.622255 12518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:09.618960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38184 (local address 127.31.87.131:36531)
0808 22:32:09.619090 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:09.619094 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:09.619109 (+ 15us) server_negotiation.cc:407] Connection header received
0808 22:32:09.619185 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:09.619187 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:09.619244 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:09.619335 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:32:09.619578 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.620070 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.620896 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.621154 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.621240 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:09.621504 (+ 264us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:32:09.621903 (+ 399us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:32:09.621998 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:09.622075 (+ 77us) server_negotiation.cc:299] Negotiation successful
0808 22:32:09.622128 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 22:32:09.627586 12151 raft_consensus.cc:1261] T 096d509446e94e628cd32452bd06565a P 35c0c3409b0145eba8283e58d3080673 [term 1 FOLLOWER]: Refusing update from remote peer a8382975b0ef4c78a8aa87bb78abda54: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:32:09.627591 11932 raft_consensus.cc:1261] T 096d509446e94e628cd32452bd06565a P c2b3097001874d8ab050b0b6f8823a76 [term 1 FOLLOWER]: Refusing update from remote peer a8382975b0ef4c78a8aa87bb78abda54: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:32:09.628225 12536 consensus_queue.cc:1035] T 096d509446e94e628cd32452bd06565a P a8382975b0ef4c78a8aa87bb78abda54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:32:09.628396 12532 consensus_queue.cc:1035] T 096d509446e94e628cd32452bd06565a P a8382975b0ef4c78a8aa87bb78abda54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:32:09.635700 12546 mvcc.cc:205] Tried to move back new op lower bound from 7316391238148472832 to 7316391237913067520. Current Snapshot: MvccSnapshot[applied={T|T < 7316391238148472832}]
I20260808 22:32:09.636879 12544 mvcc.cc:205] Tried to move back new op lower bound from 7316391238148472832 to 7316391237913067520. Current Snapshot: MvccSnapshot[applied={T|T < 7316391238148472832}]
I20260808 22:32:09.644024 11670 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33820:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:32:09.644158 11670 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33820:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:32:09.646253 11670 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9f31f95f36104c6fa687563b1fdb4d24: Sending DeleteTablet for 3 replicas of tablet 096d509446e94e628cd32452bd06565a
I20260808 22:32:09.646960 12121 tablet_service.cc:1514] Processing DeleteTablet for tablet 096d509446e94e628cd32452bd06565a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:32:09 UTC) from {username='oryx', principal='oryx/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:34662
I20260808 22:32:09.647106 12360 tablet_service.cc:1514] Processing DeleteTablet for tablet 096d509446e94e628cd32452bd06565a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:32:09 UTC) from {username='oryx', principal='oryx/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:38168
I20260808 22:32:09.647615 11900 tablet_service.cc:1514] Processing DeleteTablet for tablet 096d509446e94e628cd32452bd06565a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:32:09 UTC) from {username='oryx', principal='oryx/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:57510
I20260808 22:32:09.648003 11670 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33820:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:32:09.648355 11670 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:32:09.652436 11900 tablet_service.cc:1467] Processing CreateTablet for tablet b660396654e1494bb01f4833223c21d7 (DEFAULT_TABLE table=test-table [id=e8a4f83121d54f91a14f6bcee7b4630a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:32:09.652720 11900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b660396654e1494bb01f4833223c21d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:32:09.652676 12360 tablet_service.cc:1467] Processing CreateTablet for tablet b660396654e1494bb01f4833223c21d7 (DEFAULT_TABLE table=test-table [id=e8a4f83121d54f91a14f6bcee7b4630a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:32:09.652921 12360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b660396654e1494bb01f4833223c21d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:32:09.653439 12121 tablet_service.cc:1467] Processing CreateTablet for tablet b660396654e1494bb01f4833223c21d7 (DEFAULT_TABLE table=test-table [id=e8a4f83121d54f91a14f6bcee7b4630a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:32:09.653673 12121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b660396654e1494bb01f4833223c21d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:32:09.654845 12555 tablet_replica.cc:330] T 096d509446e94e628cd32452bd06565a P 35c0c3409b0145eba8283e58d3080673: stopping tablet replica
I20260808 22:32:09.654948 12557 tablet_replica.cc:330] T 096d509446e94e628cd32452bd06565a P c2b3097001874d8ab050b0b6f8823a76: stopping tablet replica
I20260808 22:32:09.655192 12555 raft_consensus.cc:2229] T 096d509446e94e628cd32452bd06565a P 35c0c3409b0145eba8283e58d3080673 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:32:09.655515 12556 tablet_replica.cc:330] T 096d509446e94e628cd32452bd06565a P a8382975b0ef4c78a8aa87bb78abda54: stopping tablet replica
I20260808 22:32:09.655552 12521 tablet_bootstrap.cc:492] T b660396654e1494bb01f4833223c21d7 P c2b3097001874d8ab050b0b6f8823a76: Bootstrap starting.
I20260808 22:32:09.655776 12556 raft_consensus.cc:2229] T 096d509446e94e628cd32452bd06565a P a8382975b0ef4c78a8aa87bb78abda54 [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:32:09.656172 12556 raft_consensus.cc:2258] T 096d509446e94e628cd32452bd06565a P a8382975b0ef4c78a8aa87bb78abda54 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:32:09.656510 12521 tablet_bootstrap.cc:654] T b660396654e1494bb01f4833223c21d7 P c2b3097001874d8ab050b0b6f8823a76: Neither blocks nor log segments found. Creating new log.
I20260808 22:32:09.656549 12555 raft_consensus.cc:2258] T 096d509446e94e628cd32452bd06565a P 35c0c3409b0145eba8283e58d3080673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:32:09.657619 12519 tablet_bootstrap.cc:492] T b660396654e1494bb01f4833223c21d7 P 35c0c3409b0145eba8283e58d3080673: Bootstrap starting.
I20260808 22:32:09.658253 12521 tablet_bootstrap.cc:492] T b660396654e1494bb01f4833223c21d7 P c2b3097001874d8ab050b0b6f8823a76: No bootstrap required, opened a new log
I20260808 22:32:09.658349 12521 ts_tablet_manager.cc:1397] T b660396654e1494bb01f4833223c21d7 P c2b3097001874d8ab050b0b6f8823a76: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 22:32:09.658497 12519 tablet_bootstrap.cc:654] T b660396654e1494bb01f4833223c21d7 P 35c0c3409b0145eba8283e58d3080673: Neither blocks nor log segments found. Creating new log.
I20260808 22:32:09.658737 12521 raft_consensus.cc:348] T b660396654e1494bb01f4833223c21d7 P c2b3097001874d8ab050b0b6f8823a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } } peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } }
I20260808 22:32:09.658838 12521 raft_consensus.cc:374] T b660396654e1494bb01f4833223c21d7 P c2b3097001874d8ab050b0b6f8823a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:32:09.658869 12521 raft_consensus.cc:729] T b660396654e1494bb01f4833223c21d7 P c2b3097001874d8ab050b0b6f8823a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2b3097001874d8ab050b0b6f8823a76, State: Initialized, Role: FOLLOWER
I20260808 22:32:09.658981 12521 consensus_queue.cc:260] T b660396654e1494bb01f4833223c21d7 P c2b3097001874d8ab050b0b6f8823a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } } peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } }
I20260808 22:32:09.659220 12521 ts_tablet_manager.cc:1428] T b660396654e1494bb01f4833223c21d7 P c2b3097001874d8ab050b0b6f8823a76: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 22:32:09.659412 12523 tablet_bootstrap.cc:492] T b660396654e1494bb01f4833223c21d7 P a8382975b0ef4c78a8aa87bb78abda54: Bootstrap starting.
I20260808 22:32:09.660327 12523 tablet_bootstrap.cc:654] T b660396654e1494bb01f4833223c21d7 P a8382975b0ef4c78a8aa87bb78abda54: Neither blocks nor log segments found. Creating new log.
I20260808 22:32:09.661238 12557 raft_consensus.cc:2229] T 096d509446e94e628cd32452bd06565a P c2b3097001874d8ab050b0b6f8823a76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:32:09.661528 12557 raft_consensus.cc:2258] T 096d509446e94e628cd32452bd06565a P c2b3097001874d8ab050b0b6f8823a76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:32:09.661839 12523 tablet_bootstrap.cc:492] T b660396654e1494bb01f4833223c21d7 P a8382975b0ef4c78a8aa87bb78abda54: No bootstrap required, opened a new log
I20260808 22:32:09.661917 12523 ts_tablet_manager.cc:1397] T b660396654e1494bb01f4833223c21d7 P a8382975b0ef4c78a8aa87bb78abda54: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 22:32:09.662211 12556 ts_tablet_manager.cc:1905] T 096d509446e94e628cd32452bd06565a P a8382975b0ef4c78a8aa87bb78abda54: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:32:09.662348 12523 raft_consensus.cc:348] T b660396654e1494bb01f4833223c21d7 P a8382975b0ef4c78a8aa87bb78abda54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } } peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } }
I20260808 22:32:09.662436 12519 tablet_bootstrap.cc:492] T b660396654e1494bb01f4833223c21d7 P 35c0c3409b0145eba8283e58d3080673: No bootstrap required, opened a new log
I20260808 22:32:09.662448 12523 raft_consensus.cc:374] T b660396654e1494bb01f4833223c21d7 P a8382975b0ef4c78a8aa87bb78abda54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:32:09.662482 12523 raft_consensus.cc:729] T b660396654e1494bb01f4833223c21d7 P a8382975b0ef4c78a8aa87bb78abda54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8382975b0ef4c78a8aa87bb78abda54, State: Initialized, Role: FOLLOWER
I20260808 22:32:09.662513 12519 ts_tablet_manager.cc:1397] T b660396654e1494bb01f4833223c21d7 P 35c0c3409b0145eba8283e58d3080673: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260808 22:32:09.662592 12523 consensus_queue.cc:260] T b660396654e1494bb01f4833223c21d7 P a8382975b0ef4c78a8aa87bb78abda54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } } peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } }
I20260808 22:32:09.662849 12523 ts_tablet_manager.cc:1428] T b660396654e1494bb01f4833223c21d7 P a8382975b0ef4c78a8aa87bb78abda54: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 22:32:09.662987 12519 raft_consensus.cc:348] T b660396654e1494bb01f4833223c21d7 P 35c0c3409b0145eba8283e58d3080673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } } peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } }
I20260808 22:32:09.663098 12519 raft_consensus.cc:374] T b660396654e1494bb01f4833223c21d7 P 35c0c3409b0145eba8283e58d3080673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:32:09.663141 12519 raft_consensus.cc:729] T b660396654e1494bb01f4833223c21d7 P 35c0c3409b0145eba8283e58d3080673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35c0c3409b0145eba8283e58d3080673, State: Initialized, Role: FOLLOWER
I20260808 22:32:09.663259 12519 consensus_queue.cc:260] T b660396654e1494bb01f4833223c21d7 P 35c0c3409b0145eba8283e58d3080673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } } peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } }
I20260808 22:32:09.663627 12519 ts_tablet_manager.cc:1428] T b660396654e1494bb01f4833223c21d7 P 35c0c3409b0145eba8283e58d3080673: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 22:32:09.664003 12555 ts_tablet_manager.cc:1905] T 096d509446e94e628cd32452bd06565a P 35c0c3409b0145eba8283e58d3080673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:32:09.666271 12557 ts_tablet_manager.cc:1905] T 096d509446e94e628cd32452bd06565a P c2b3097001874d8ab050b0b6f8823a76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:32:09.666466 12555 ts_tablet_manager.cc:1918] T 096d509446e94e628cd32452bd06565a P 35c0c3409b0145eba8283e58d3080673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:32:09.666563 12555 log.cc:1196] T 096d509446e94e628cd32452bd06565a P 35c0c3409b0145eba8283e58d3080673: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-1/wal/wals/096d509446e94e628cd32452bd06565a
I20260808 22:32:09.668465 12557 ts_tablet_manager.cc:1918] T 096d509446e94e628cd32452bd06565a P c2b3097001874d8ab050b0b6f8823a76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:32:09.668545 12557 log.cc:1196] T 096d509446e94e628cd32452bd06565a P c2b3097001874d8ab050b0b6f8823a76: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-0/wal/wals/096d509446e94e628cd32452bd06565a
I20260808 22:32:09.668737 12556 ts_tablet_manager.cc:1918] T 096d509446e94e628cd32452bd06565a P a8382975b0ef4c78a8aa87bb78abda54: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:32:09.668803 12557 ts_tablet_manager.cc:1939] T 096d509446e94e628cd32452bd06565a P c2b3097001874d8ab050b0b6f8823a76: Deleting consensus metadata
I20260808 22:32:09.668829 12556 log.cc:1196] T 096d509446e94e628cd32452bd06565a P a8382975b0ef4c78a8aa87bb78abda54: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-2/wal/wals/096d509446e94e628cd32452bd06565a
I20260808 22:32:09.669129 12556 ts_tablet_manager.cc:1939] T 096d509446e94e628cd32452bd06565a P a8382975b0ef4c78a8aa87bb78abda54: Deleting consensus metadata
I20260808 22:32:09.669950 11654 catalog_manager.cc:4841] TS c2b3097001874d8ab050b0b6f8823a76 (127.31.87.129:40109): tablet 096d509446e94e628cd32452bd06565a (table test-table [id=35578e809d084038b4a8258fbeb791da]) successfully deleted
I20260808 22:32:09.670176 12555 ts_tablet_manager.cc:1939] T 096d509446e94e628cd32452bd06565a P 35c0c3409b0145eba8283e58d3080673: Deleting consensus metadata
I20260808 22:32:09.670186 11650 catalog_manager.cc:4841] TS a8382975b0ef4c78a8aa87bb78abda54 (127.31.87.131:36531): tablet 096d509446e94e628cd32452bd06565a (table test-table [id=35578e809d084038b4a8258fbeb791da]) successfully deleted
I20260808 22:32:09.671207 11650 catalog_manager.cc:4841] TS 35c0c3409b0145eba8283e58d3080673 (127.31.87.130:41999): tablet 096d509446e94e628cd32452bd06565a (table test-table [id=35578e809d084038b4a8258fbeb791da]) successfully deleted
I20260808 22:32:09.712755 12530 raft_consensus.cc:482] T b660396654e1494bb01f4833223c21d7 P 35c0c3409b0145eba8283e58d3080673 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:32:09.712888 12530 raft_consensus.cc:504] T b660396654e1494bb01f4833223c21d7 P 35c0c3409b0145eba8283e58d3080673 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } } peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } }
I20260808 22:32:09.713945 12530 leader_election.cc:288] T b660396654e1494bb01f4833223c21d7 P 35c0c3409b0145eba8283e58d3080673 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2b3097001874d8ab050b0b6f8823a76 (127.31.87.129:40109), a8382975b0ef4c78a8aa87bb78abda54 (127.31.87.131:36531)
I20260808 22:32:09.718350 12518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:09.714300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:37423 (local address 127.31.87.131:36531)
0808 22:32:09.714421 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:09.714426 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:09.714801 (+ 375us) server_negotiation.cc:407] Connection header received
0808 22:32:09.714988 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:09.714992 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:09.715038 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:09.715102 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:32:09.715491 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.716190 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.717397 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.717811 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.717924 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:09.717979 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:09.718158 (+ 179us) server_negotiation.cc:299] Negotiation successful
0808 22:32:09.718224 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 22:32:09.718375 12562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:09.714166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.131:36531 (local address 127.31.87.130:37423)
0808 22:32:09.714731 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:09.714742 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:09.714853 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:09.715209 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:09.715212 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:09.715231 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:32:09.715373 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:09.715379 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.716329 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.716332 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:09.717279 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:09.717289 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.717921 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.717924 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:09.718026 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:09.718053 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 22:32:09.718138 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 22:32:09.718208 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"mutex_wait_us":190,"thread_start_us":107,"threads_started":1}
I20260808 22:32:09.719043 12563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:09.714165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:40109 (local address 127.31.87.130:47407)
0808 22:32:09.714683 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:09.714697 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:09.714822 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:09.715083 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:09.715085 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:09.715104 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:32:09.715258 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:09.715272 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.716280 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.716284 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:09.717654 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:09.717665 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.718648 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.718651 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:09.718750 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:09.718773 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 22:32:09.718845 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 22:32:09.718907 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"mutex_wait_us":38,"thread_start_us":68,"threads_started":1}
I20260808 22:32:09.719041 12517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:09.714472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:47407 (local address 127.31.87.129:40109)
0808 22:32:09.714593 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:09.714596 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:09.714784 (+ 188us) server_negotiation.cc:407] Connection header received
0808 22:32:09.714929 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:09.714933 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:09.714983 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:09.715064 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:32:09.715379 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.716167 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.717785 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.718288 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.718399 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:09.718444 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:09.718865 (+ 421us) server_negotiation.cc:299] Negotiation successful
0808 22:32:09.718940 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 22:32:09.718981 12395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b660396654e1494bb01f4833223c21d7" candidate_uuid: "35c0c3409b0145eba8283e58d3080673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8382975b0ef4c78a8aa87bb78abda54" is_pre_election: true
I20260808 22:32:09.719234 12395 raft_consensus.cc:2454] T b660396654e1494bb01f4833223c21d7 P a8382975b0ef4c78a8aa87bb78abda54 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35c0c3409b0145eba8283e58d3080673 in term 0.
I20260808 22:32:09.719425 11932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b660396654e1494bb01f4833223c21d7" candidate_uuid: "35c0c3409b0145eba8283e58d3080673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2b3097001874d8ab050b0b6f8823a76" is_pre_election: true
I20260808 22:32:09.719560 11932 raft_consensus.cc:2454] T b660396654e1494bb01f4833223c21d7 P c2b3097001874d8ab050b0b6f8823a76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35c0c3409b0145eba8283e58d3080673 in term 0.
I20260808 22:32:09.719827 12040 leader_election.cc:302] T b660396654e1494bb01f4833223c21d7 P 35c0c3409b0145eba8283e58d3080673 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35c0c3409b0145eba8283e58d3080673, a8382975b0ef4c78a8aa87bb78abda54; no voters:
I20260808 22:32:09.720098 12530 raft_consensus.cc:2781] T b660396654e1494bb01f4833223c21d7 P 35c0c3409b0145eba8283e58d3080673 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:32:09.720156 12530 raft_consensus.cc:482] T b660396654e1494bb01f4833223c21d7 P 35c0c3409b0145eba8283e58d3080673 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:32:09.720199 12530 raft_consensus.cc:3037] T b660396654e1494bb01f4833223c21d7 P 35c0c3409b0145eba8283e58d3080673 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:32:09.721169 12530 raft_consensus.cc:504] T b660396654e1494bb01f4833223c21d7 P 35c0c3409b0145eba8283e58d3080673 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } } peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } }
I20260808 22:32:09.721503 12530 leader_election.cc:288] T b660396654e1494bb01f4833223c21d7 P 35c0c3409b0145eba8283e58d3080673 [CANDIDATE]: Term 1 election: Requested vote from peers c2b3097001874d8ab050b0b6f8823a76 (127.31.87.129:40109), a8382975b0ef4c78a8aa87bb78abda54 (127.31.87.131:36531)
I20260808 22:32:09.721832 11932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b660396654e1494bb01f4833223c21d7" candidate_uuid: "35c0c3409b0145eba8283e58d3080673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2b3097001874d8ab050b0b6f8823a76"
I20260808 22:32:09.721845 12395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b660396654e1494bb01f4833223c21d7" candidate_uuid: "35c0c3409b0145eba8283e58d3080673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8382975b0ef4c78a8aa87bb78abda54"
I20260808 22:32:09.721920 11932 raft_consensus.cc:3037] T b660396654e1494bb01f4833223c21d7 P c2b3097001874d8ab050b0b6f8823a76 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:32:09.721956 12395 raft_consensus.cc:3037] T b660396654e1494bb01f4833223c21d7 P a8382975b0ef4c78a8aa87bb78abda54 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:32:09.722827 11932 raft_consensus.cc:2454] T b660396654e1494bb01f4833223c21d7 P c2b3097001874d8ab050b0b6f8823a76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35c0c3409b0145eba8283e58d3080673 in term 1.
I20260808 22:32:09.723126 12046 leader_election.cc:302] T b660396654e1494bb01f4833223c21d7 P 35c0c3409b0145eba8283e58d3080673 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35c0c3409b0145eba8283e58d3080673, c2b3097001874d8ab050b0b6f8823a76; no voters:
I20260808 22:32:09.723129 12395 raft_consensus.cc:2454] T b660396654e1494bb01f4833223c21d7 P a8382975b0ef4c78a8aa87bb78abda54 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35c0c3409b0145eba8283e58d3080673 in term 1.
I20260808 22:32:09.723557 12530 raft_consensus.cc:2781] T b660396654e1494bb01f4833223c21d7 P 35c0c3409b0145eba8283e58d3080673 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:32:09.723752 12530 raft_consensus.cc:686] T b660396654e1494bb01f4833223c21d7 P 35c0c3409b0145eba8283e58d3080673 [term 1 LEADER]: Becoming Leader. State: Replica: 35c0c3409b0145eba8283e58d3080673, State: Running, Role: LEADER
I20260808 22:32:09.724009 12530 consensus_queue.cc:237] T b660396654e1494bb01f4833223c21d7 P 35c0c3409b0145eba8283e58d3080673 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } } peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } }
I20260808 22:32:09.726300 11670 catalog_manager.cc:5495] T b660396654e1494bb01f4833223c21d7 P 35c0c3409b0145eba8283e58d3080673 reported cstate change: term changed from 0 to 1, leader changed from <none> to 35c0c3409b0145eba8283e58d3080673 (127.31.87.130). New cstate: current_term: 1 leader_uuid: "35c0c3409b0145eba8283e58d3080673" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } health_report { overall_health: UNKNOWN } } }
Aug 08 22:32:09 dist-test-slave-266d krb5kdc[11600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228328, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.87.190@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM
I20260808 22:32:09.767395 11748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:09.758625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36908 (local address 127.31.87.190:45745)
0808 22:32:09.758753 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:09.758757 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:09.758883 (+ 126us) server_negotiation.cc:407] Connection header received
0808 22:32:09.758985 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:09.758989 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:09.759038 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:09.759148 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:09.759717 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.760516 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.761289 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.761626 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.761768 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:09.764329 (+ 2561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:09.764332 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:09.764334 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:09.764355 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:09.765622 (+ 1267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:09.766155 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:09.766157 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:09.766159 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:09.766220 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:09.766487 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:09.766489 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:09.766490 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:09.766636 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:09.766701 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:09.767031 (+ 330us) server_negotiation.cc:299] Negotiation successful
0808 22:32:09.767222 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":101,"server-negotiator.queue_time_us":55}
I20260808 22:32:09.767428 12579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:09.758522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45745 (local address 127.0.0.1:36908)
0808 22:32:09.758800 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:09.758814 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:09.758906 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:09.759158 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:09.759160 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:09.759507 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:09.759601 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:09.759607 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.760612 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.760614 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:09.761177 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:09.761184 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.761738 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.761740 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:09.761833 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:09.762384 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:09.762396 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:09.764199 (+ 1803us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:32:09.765731 (+ 1532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:09.765742 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:09.765748 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:09.766058 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:09.766300 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:09.766303 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:09.766305 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:09.766408 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:09.766735 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:32:09.766742 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:32:09.766869 (+ 127us) client_negotiation.cc:769] Sending connection context
0808 22:32:09.766994 (+ 125us) client_negotiation.cc:240] Negotiation successful
0808 22:32:09.767180 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":18,"thread_start_us":90,"threads_started":1}
I20260808 22:32:09.771056 11670 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:36908:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 22:32:09.771718 11670 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:32:09.776367 11900 tablet_service.cc:1467] Processing CreateTablet for tablet b575234acfe94882ba066121d0d3dd37 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=58587299a9844405a075cce31e87b24e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 22:32:09.776531 12121 tablet_service.cc:1467] Processing CreateTablet for tablet b575234acfe94882ba066121d0d3dd37 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=58587299a9844405a075cce31e87b24e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 22:32:09.776790 11900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b575234acfe94882ba066121d0d3dd37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:32:09.776904 12121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b575234acfe94882ba066121d0d3dd37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:32:09.779809 12519 tablet_bootstrap.cc:492] T b575234acfe94882ba066121d0d3dd37 P 35c0c3409b0145eba8283e58d3080673: Bootstrap starting.
I20260808 22:32:09.779999 12521 tablet_bootstrap.cc:492] T b575234acfe94882ba066121d0d3dd37 P c2b3097001874d8ab050b0b6f8823a76: Bootstrap starting.
I20260808 22:32:09.780212 12360 tablet_service.cc:1467] Processing CreateTablet for tablet b575234acfe94882ba066121d0d3dd37 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=58587299a9844405a075cce31e87b24e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 22:32:09.780562 12360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b575234acfe94882ba066121d0d3dd37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:32:09.780848 12519 tablet_bootstrap.cc:654] T b575234acfe94882ba066121d0d3dd37 P 35c0c3409b0145eba8283e58d3080673: Neither blocks nor log segments found. Creating new log.
I20260808 22:32:09.781023 12521 tablet_bootstrap.cc:654] T b575234acfe94882ba066121d0d3dd37 P c2b3097001874d8ab050b0b6f8823a76: Neither blocks nor log segments found. Creating new log.
I20260808 22:32:09.782711 12521 tablet_bootstrap.cc:492] T b575234acfe94882ba066121d0d3dd37 P c2b3097001874d8ab050b0b6f8823a76: No bootstrap required, opened a new log
I20260808 22:32:09.782778 12521 ts_tablet_manager.cc:1397] T b575234acfe94882ba066121d0d3dd37 P c2b3097001874d8ab050b0b6f8823a76: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 22:32:09.783167 12523 tablet_bootstrap.cc:492] T b575234acfe94882ba066121d0d3dd37 P a8382975b0ef4c78a8aa87bb78abda54: Bootstrap starting.
I20260808 22:32:09.783174 12521 raft_consensus.cc:348] T b575234acfe94882ba066121d0d3dd37 P c2b3097001874d8ab050b0b6f8823a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } } peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } }
I20260808 22:32:09.783272 12521 raft_consensus.cc:374] T b575234acfe94882ba066121d0d3dd37 P c2b3097001874d8ab050b0b6f8823a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:32:09.783298 12521 raft_consensus.cc:729] T b575234acfe94882ba066121d0d3dd37 P c2b3097001874d8ab050b0b6f8823a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2b3097001874d8ab050b0b6f8823a76, State: Initialized, Role: FOLLOWER
I20260808 22:32:09.783404 12521 consensus_queue.cc:260] T b575234acfe94882ba066121d0d3dd37 P c2b3097001874d8ab050b0b6f8823a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } } peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } }
I20260808 22:32:09.783632 12521 ts_tablet_manager.cc:1428] T b575234acfe94882ba066121d0d3dd37 P c2b3097001874d8ab050b0b6f8823a76: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 22:32:09.783828 12534 tablet_replica.cc:439] T b575234acfe94882ba066121d0d3dd37 P c2b3097001874d8ab050b0b6f8823a76: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } } peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } } }
I20260808 22:32:09.783941 12534 tablet_replica.cc:442] T b575234acfe94882ba066121d0d3dd37 P c2b3097001874d8ab050b0b6f8823a76: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:32:09.784179 12523 tablet_bootstrap.cc:654] T b575234acfe94882ba066121d0d3dd37 P a8382975b0ef4c78a8aa87bb78abda54: Neither blocks nor log segments found. Creating new log.
I20260808 22:32:09.784895 12519 tablet_bootstrap.cc:492] T b575234acfe94882ba066121d0d3dd37 P 35c0c3409b0145eba8283e58d3080673: No bootstrap required, opened a new log
I20260808 22:32:09.784979 12519 ts_tablet_manager.cc:1397] T b575234acfe94882ba066121d0d3dd37 P 35c0c3409b0145eba8283e58d3080673: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260808 22:32:09.785439 12519 raft_consensus.cc:348] T b575234acfe94882ba066121d0d3dd37 P 35c0c3409b0145eba8283e58d3080673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } } peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } }
I20260808 22:32:09.785547 12519 raft_consensus.cc:374] T b575234acfe94882ba066121d0d3dd37 P 35c0c3409b0145eba8283e58d3080673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:32:09.785575 12519 raft_consensus.cc:729] T b575234acfe94882ba066121d0d3dd37 P 35c0c3409b0145eba8283e58d3080673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35c0c3409b0145eba8283e58d3080673, State: Initialized, Role: FOLLOWER
I20260808 22:32:09.785681 12519 consensus_queue.cc:260] T b575234acfe94882ba066121d0d3dd37 P 35c0c3409b0145eba8283e58d3080673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } } peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } }
I20260808 22:32:09.785902 12519 ts_tablet_manager.cc:1428] T b575234acfe94882ba066121d0d3dd37 P 35c0c3409b0145eba8283e58d3080673: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 22:32:09.786053 12530 tablet_replica.cc:439] T b575234acfe94882ba066121d0d3dd37 P 35c0c3409b0145eba8283e58d3080673: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } } peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } } }
I20260808 22:32:09.786170 12530 tablet_replica.cc:442] T b575234acfe94882ba066121d0d3dd37 P 35c0c3409b0145eba8283e58d3080673: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:32:09.786300 12523 tablet_bootstrap.cc:492] T b575234acfe94882ba066121d0d3dd37 P a8382975b0ef4c78a8aa87bb78abda54: No bootstrap required, opened a new log
I20260808 22:32:09.786386 12523 ts_tablet_manager.cc:1397] T b575234acfe94882ba066121d0d3dd37 P a8382975b0ef4c78a8aa87bb78abda54: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 22:32:09.786794 12523 raft_consensus.cc:348] T b575234acfe94882ba066121d0d3dd37 P a8382975b0ef4c78a8aa87bb78abda54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } } peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } }
I20260808 22:32:09.786900 12523 raft_consensus.cc:374] T b575234acfe94882ba066121d0d3dd37 P a8382975b0ef4c78a8aa87bb78abda54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:32:09.786931 12523 raft_consensus.cc:729] T b575234acfe94882ba066121d0d3dd37 P a8382975b0ef4c78a8aa87bb78abda54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8382975b0ef4c78a8aa87bb78abda54, State: Initialized, Role: FOLLOWER
I20260808 22:32:09.787042 12523 consensus_queue.cc:260] T b575234acfe94882ba066121d0d3dd37 P a8382975b0ef4c78a8aa87bb78abda54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } } peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } }
I20260808 22:32:09.787297 12523 ts_tablet_manager.cc:1428] T b575234acfe94882ba066121d0d3dd37 P a8382975b0ef4c78a8aa87bb78abda54: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 22:32:09.787474 12551 tablet_replica.cc:439] T b575234acfe94882ba066121d0d3dd37 P a8382975b0ef4c78a8aa87bb78abda54: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } } peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } } }
I20260808 22:32:09.787623 12551 tablet_replica.cc:442] T b575234acfe94882ba066121d0d3dd37 P a8382975b0ef4c78a8aa87bb78abda54: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:32:09.847680 12530 raft_consensus.cc:482] T b575234acfe94882ba066121d0d3dd37 P 35c0c3409b0145eba8283e58d3080673 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:32:09.847816 12530 raft_consensus.cc:504] T b575234acfe94882ba066121d0d3dd37 P 35c0c3409b0145eba8283e58d3080673 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } } peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } }
I20260808 22:32:09.848269 12530 leader_election.cc:288] T b575234acfe94882ba066121d0d3dd37 P 35c0c3409b0145eba8283e58d3080673 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2b3097001874d8ab050b0b6f8823a76 (127.31.87.129:40109), a8382975b0ef4c78a8aa87bb78abda54 (127.31.87.131:36531)
I20260808 22:32:09.848624 12395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b575234acfe94882ba066121d0d3dd37" candidate_uuid: "35c0c3409b0145eba8283e58d3080673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8382975b0ef4c78a8aa87bb78abda54" is_pre_election: true
I20260808 22:32:09.848773 12395 raft_consensus.cc:2454] T b575234acfe94882ba066121d0d3dd37 P a8382975b0ef4c78a8aa87bb78abda54 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35c0c3409b0145eba8283e58d3080673 in term 0.
I20260808 22:32:09.849040 11932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b575234acfe94882ba066121d0d3dd37" candidate_uuid: "35c0c3409b0145eba8283e58d3080673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2b3097001874d8ab050b0b6f8823a76" is_pre_election: true
I20260808 22:32:09.849166 12040 leader_election.cc:302] T b575234acfe94882ba066121d0d3dd37 P 35c0c3409b0145eba8283e58d3080673 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35c0c3409b0145eba8283e58d3080673, a8382975b0ef4c78a8aa87bb78abda54; no voters:
I20260808 22:32:09.849212 11932 raft_consensus.cc:2454] T b575234acfe94882ba066121d0d3dd37 P c2b3097001874d8ab050b0b6f8823a76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35c0c3409b0145eba8283e58d3080673 in term 0.
I20260808 22:32:09.849490 12530 raft_consensus.cc:2781] T b575234acfe94882ba066121d0d3dd37 P 35c0c3409b0145eba8283e58d3080673 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:32:09.849546 12530 raft_consensus.cc:482] T b575234acfe94882ba066121d0d3dd37 P 35c0c3409b0145eba8283e58d3080673 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:32:09.849583 12530 raft_consensus.cc:3037] T b575234acfe94882ba066121d0d3dd37 P 35c0c3409b0145eba8283e58d3080673 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:32:09.850790 12530 raft_consensus.cc:504] T b575234acfe94882ba066121d0d3dd37 P 35c0c3409b0145eba8283e58d3080673 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } } peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } }
I20260808 22:32:09.851117 12530 leader_election.cc:288] T b575234acfe94882ba066121d0d3dd37 P 35c0c3409b0145eba8283e58d3080673 [CANDIDATE]: Term 1 election: Requested vote from peers c2b3097001874d8ab050b0b6f8823a76 (127.31.87.129:40109), a8382975b0ef4c78a8aa87bb78abda54 (127.31.87.131:36531)
I20260808 22:32:09.851507 12395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b575234acfe94882ba066121d0d3dd37" candidate_uuid: "35c0c3409b0145eba8283e58d3080673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8382975b0ef4c78a8aa87bb78abda54"
I20260808 22:32:09.851634 12395 raft_consensus.cc:3037] T b575234acfe94882ba066121d0d3dd37 P a8382975b0ef4c78a8aa87bb78abda54 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:32:09.852798 12395 raft_consensus.cc:2454] T b575234acfe94882ba066121d0d3dd37 P a8382975b0ef4c78a8aa87bb78abda54 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35c0c3409b0145eba8283e58d3080673 in term 1.
I20260808 22:32:09.853080 11932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b575234acfe94882ba066121d0d3dd37" candidate_uuid: "35c0c3409b0145eba8283e58d3080673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2b3097001874d8ab050b0b6f8823a76"
I20260808 22:32:09.853135 12040 leader_election.cc:302] T b575234acfe94882ba066121d0d3dd37 P 35c0c3409b0145eba8283e58d3080673 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35c0c3409b0145eba8283e58d3080673, a8382975b0ef4c78a8aa87bb78abda54; no voters:
I20260808 22:32:09.853232 11932 raft_consensus.cc:3037] T b575234acfe94882ba066121d0d3dd37 P c2b3097001874d8ab050b0b6f8823a76 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:32:09.853319 12530 raft_consensus.cc:2781] T b575234acfe94882ba066121d0d3dd37 P 35c0c3409b0145eba8283e58d3080673 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:32:09.853406 12530 raft_consensus.cc:686] T b575234acfe94882ba066121d0d3dd37 P 35c0c3409b0145eba8283e58d3080673 [term 1 LEADER]: Becoming Leader. State: Replica: 35c0c3409b0145eba8283e58d3080673, State: Running, Role: LEADER
I20260808 22:32:09.853546 12530 consensus_queue.cc:237] T b575234acfe94882ba066121d0d3dd37 P 35c0c3409b0145eba8283e58d3080673 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } } peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } }
I20260808 22:32:09.854283 12530 tablet_replica.cc:439] T b575234acfe94882ba066121d0d3dd37 P 35c0c3409b0145eba8283e58d3080673: TxnStatusTablet state changed. Reason: New leader 35c0c3409b0145eba8283e58d3080673. Latest consensus state: current_term: 1 leader_uuid: "35c0c3409b0145eba8283e58d3080673" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } } peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } } }
I20260808 22:32:09.854341 11932 raft_consensus.cc:2454] T b575234acfe94882ba066121d0d3dd37 P c2b3097001874d8ab050b0b6f8823a76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35c0c3409b0145eba8283e58d3080673 in term 1.
I20260808 22:32:09.854385 12530 tablet_replica.cc:442] T b575234acfe94882ba066121d0d3dd37 P 35c0c3409b0145eba8283e58d3080673: This TxnStatusTablet replica's current role is: LEADER
I20260808 22:32:09.854707 12587 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 22:32:09.854889 12587 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 22:32:09.855489 11670 catalog_manager.cc:5495] T b575234acfe94882ba066121d0d3dd37 P 35c0c3409b0145eba8283e58d3080673 reported cstate change: term changed from 0 to 1, leader changed from <none> to 35c0c3409b0145eba8283e58d3080673 (127.31.87.130). New cstate: current_term: 1 leader_uuid: "35c0c3409b0145eba8283e58d3080673" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } health_report { overall_health: UNKNOWN } } }
I20260808 22:32:09.883098 12579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:09.879150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.130:41999 (local address 127.0.0.1:43422)
0808 22:32:09.879343 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:09.879356 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:09.879446 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:09.879738 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:09.879741 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:09.879748 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 22:32:09.879845 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:09.879850 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.880844 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.880847 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:09.881476 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:09.881484 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.882023 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.882026 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:09.882096 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:09.882138 (+ 42us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 22:32:09.882831 (+ 693us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 22:32:09.882842 (+ 11us) client_negotiation.cc:769] Sending connection context
0808 22:32:09.882893 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 22:32:09.882983 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 22:32:09.883109 12512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:09.879234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43422 (local address 127.31.87.130:41999)
0808 22:32:09.879354 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:09.879358 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:09.879397 (+ 39us) server_negotiation.cc:407] Connection header received
0808 22:32:09.879565 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:09.879568 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:09.879632 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:09.879734 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:32:09.879931 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.880751 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.881596 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.881935 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.882034 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:09.882262 (+ 228us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:32:09.882750 (+ 488us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:32:09.882855 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:09.882907 (+ 52us) server_negotiation.cc:299] Negotiation successful
0808 22:32:09.882961 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 22:32:09.885929 11932 raft_consensus.cc:1261] T b575234acfe94882ba066121d0d3dd37 P c2b3097001874d8ab050b0b6f8823a76 [term 1 FOLLOWER]: Refusing update from remote peer 35c0c3409b0145eba8283e58d3080673: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:32:09.886051 12395 raft_consensus.cc:1261] T b575234acfe94882ba066121d0d3dd37 P a8382975b0ef4c78a8aa87bb78abda54 [term 1 FOLLOWER]: Refusing update from remote peer 35c0c3409b0145eba8283e58d3080673: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:32:09.886471 12530 consensus_queue.cc:1035] T b575234acfe94882ba066121d0d3dd37 P 35c0c3409b0145eba8283e58d3080673 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:32:09.886655 12566 consensus_queue.cc:1035] T b575234acfe94882ba066121d0d3dd37 P 35c0c3409b0145eba8283e58d3080673 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:32:09.887918 12534 tablet_replica.cc:439] T b575234acfe94882ba066121d0d3dd37 P c2b3097001874d8ab050b0b6f8823a76: TxnStatusTablet state changed. Reason: New leader 35c0c3409b0145eba8283e58d3080673. Latest consensus state: current_term: 1 leader_uuid: "35c0c3409b0145eba8283e58d3080673" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } } peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } } }
I20260808 22:32:09.888031 12534 tablet_replica.cc:442] T b575234acfe94882ba066121d0d3dd37 P c2b3097001874d8ab050b0b6f8823a76: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:32:09.888203 12551 tablet_replica.cc:439] T b575234acfe94882ba066121d0d3dd37 P a8382975b0ef4c78a8aa87bb78abda54: TxnStatusTablet state changed. Reason: New leader 35c0c3409b0145eba8283e58d3080673. Latest consensus state: current_term: 1 leader_uuid: "35c0c3409b0145eba8283e58d3080673" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } } peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } } }
I20260808 22:32:09.888299 12551 tablet_replica.cc:442] T b575234acfe94882ba066121d0d3dd37 P a8382975b0ef4c78a8aa87bb78abda54: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:32:09.889703 12566 tablet_replica.cc:439] T b575234acfe94882ba066121d0d3dd37 P 35c0c3409b0145eba8283e58d3080673: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "35c0c3409b0145eba8283e58d3080673" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } } peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } } }
I20260808 22:32:09.889796 12566 tablet_replica.cc:442] T b575234acfe94882ba066121d0d3dd37 P 35c0c3409b0145eba8283e58d3080673: This TxnStatusTablet replica's current role is: LEADER
I20260808 22:32:09.890612 12534 tablet_replica.cc:439] T b575234acfe94882ba066121d0d3dd37 P c2b3097001874d8ab050b0b6f8823a76: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "35c0c3409b0145eba8283e58d3080673" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } } peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } } }
I20260808 22:32:09.890703 12534 tablet_replica.cc:442] T b575234acfe94882ba066121d0d3dd37 P c2b3097001874d8ab050b0b6f8823a76: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:32:09.894174 12530 tablet_replica.cc:439] T b575234acfe94882ba066121d0d3dd37 P 35c0c3409b0145eba8283e58d3080673: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "35c0c3409b0145eba8283e58d3080673" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } } peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } } }
I20260808 22:32:09.894285 12530 tablet_replica.cc:442] T b575234acfe94882ba066121d0d3dd37 P 35c0c3409b0145eba8283e58d3080673: This TxnStatusTablet replica's current role is: LEADER
I20260808 22:32:09.895396 12541 mvcc.cc:205] Tried to move back new op lower bound from 7316391239207346176 to 7316391239081517056. Current Snapshot: MvccSnapshot[applied={T|T < 7316391239207346176}]
I20260808 22:32:09.895566 12551 tablet_replica.cc:439] T b575234acfe94882ba066121d0d3dd37 P a8382975b0ef4c78a8aa87bb78abda54: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "35c0c3409b0145eba8283e58d3080673" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } } peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } } }
I20260808 22:32:09.895653 12551 tablet_replica.cc:442] T b575234acfe94882ba066121d0d3dd37 P a8382975b0ef4c78a8aa87bb78abda54: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:32:09.898151 12512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:09.894685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43426 (local address 127.31.87.130:41999)
0808 22:32:09.894784 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:09.894788 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:09.895000 (+ 212us) server_negotiation.cc:407] Connection header received
0808 22:32:09.895042 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:09.895045 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:09.895086 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:09.895146 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:32:09.895486 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.896134 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.897317 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.897533 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.897608 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:09.897840 (+ 232us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:32:09.897894 (+ 54us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:32:09.897937 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:09.898026 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 22:32:09.898061 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260808 22:32:09.905827 12001 tablet.cc:2369] T b660396654e1494bb01f4833223c21d7 P c2b3097001874d8ab050b0b6f8823a76: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:32:09.913856 12512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:09.909973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:38917 (local address 127.31.87.130:41999)
0808 22:32:09.910059 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:09.910063 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:09.910349 (+ 286us) server_negotiation.cc:407] Connection header received
0808 22:32:09.910472 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:09.910475 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:09.910514 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:09.910582 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:32:09.910808 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.911525 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.912683 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.913010 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.913132 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:09.913451 (+ 319us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:32:09.913519 (+ 68us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:32:09.913582 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:09.913669 (+ 87us) server_negotiation.cc:299] Negotiation successful
0808 22:32:09.913725 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":31}
I20260808 22:32:09.914085 12596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:09.909942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.130:41999 (local address 127.31.87.130:38917)
0808 22:32:09.910272 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:09.910285 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:09.910392 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:09.910598 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:09.910601 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:09.910608 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 22:32:09.910727 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:09.910733 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.911614 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.911617 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:09.912575 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:09.912584 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.913140 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.913143 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:09.913284 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:09.913331 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 22:32:09.913593 (+ 262us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 22:32:09.913599 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 22:32:09.913654 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 22:32:09.913749 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"mutex_wait_us":49,"thread_start_us":83,"threads_started":1}
I20260808 22:32:09.919889 12394 raft_consensus.cc:1261] T b660396654e1494bb01f4833223c21d7 P a8382975b0ef4c78a8aa87bb78abda54 [term 1 FOLLOWER]: Refusing update from remote peer 35c0c3409b0145eba8283e58d3080673: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:32:09.920370 11932 raft_consensus.cc:1261] T b660396654e1494bb01f4833223c21d7 P c2b3097001874d8ab050b0b6f8823a76 [term 1 FOLLOWER]: Refusing update from remote peer 35c0c3409b0145eba8283e58d3080673: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:32:09.920342 12593 consensus_queue.cc:1035] T b660396654e1494bb01f4833223c21d7 P 35c0c3409b0145eba8283e58d3080673 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:32:09.920735 12530 consensus_queue.cc:1035] T b660396654e1494bb01f4833223c21d7 P 35c0c3409b0145eba8283e58d3080673 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:32:09.958361 11670 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33820:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:32:09.958500 11670 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33820:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:32:09.960325 11670 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9f31f95f36104c6fa687563b1fdb4d24: Sending DeleteTablet for 3 replicas of tablet b660396654e1494bb01f4833223c21d7
I20260808 22:32:09.960863 11900 tablet_service.cc:1514] Processing DeleteTablet for tablet b660396654e1494bb01f4833223c21d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:32:09 UTC) from {username='oryx', principal='oryx/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:57510
I20260808 22:32:09.961058 12360 tablet_service.cc:1514] Processing DeleteTablet for tablet b660396654e1494bb01f4833223c21d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:32:09 UTC) from {username='oryx', principal='oryx/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:38168
I20260808 22:32:09.961264 12557 tablet_replica.cc:330] T b660396654e1494bb01f4833223c21d7 P c2b3097001874d8ab050b0b6f8823a76: stopping tablet replica
I20260808 22:32:09.961254 12120 tablet_service.cc:1514] Processing DeleteTablet for tablet b660396654e1494bb01f4833223c21d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:32:09 UTC) from {username='oryx', principal='oryx/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:34662
I20260808 22:32:09.961390 12557 raft_consensus.cc:2229] T b660396654e1494bb01f4833223c21d7 P c2b3097001874d8ab050b0b6f8823a76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:32:09.961418 12555 tablet_replica.cc:330] T b660396654e1494bb01f4833223c21d7 P 35c0c3409b0145eba8283e58d3080673: stopping tablet replica
I20260808 22:32:09.961501 12557 raft_consensus.cc:2258] T b660396654e1494bb01f4833223c21d7 P c2b3097001874d8ab050b0b6f8823a76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:32:09.961511 12555 raft_consensus.cc:2229] T b660396654e1494bb01f4833223c21d7 P 35c0c3409b0145eba8283e58d3080673 [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:32:09.961659 12556 tablet_replica.cc:330] T b660396654e1494bb01f4833223c21d7 P a8382975b0ef4c78a8aa87bb78abda54: stopping tablet replica
I20260808 22:32:09.961779 12556 raft_consensus.cc:2229] T b660396654e1494bb01f4833223c21d7 P a8382975b0ef4c78a8aa87bb78abda54 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:32:09.961802 12555 raft_consensus.cc:2258] T b660396654e1494bb01f4833223c21d7 P 35c0c3409b0145eba8283e58d3080673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:32:09.961889 12556 raft_consensus.cc:2258] T b660396654e1494bb01f4833223c21d7 P a8382975b0ef4c78a8aa87bb78abda54 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:32:09.962687 12557 ts_tablet_manager.cc:1905] T b660396654e1494bb01f4833223c21d7 P c2b3097001874d8ab050b0b6f8823a76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:32:09.962896 12555 ts_tablet_manager.cc:1905] T b660396654e1494bb01f4833223c21d7 P 35c0c3409b0145eba8283e58d3080673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:32:09.965504 12557 ts_tablet_manager.cc:1918] T b660396654e1494bb01f4833223c21d7 P c2b3097001874d8ab050b0b6f8823a76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 22:32:09.965581 12557 log.cc:1196] T b660396654e1494bb01f4833223c21d7 P c2b3097001874d8ab050b0b6f8823a76: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-0/wal/wals/b660396654e1494bb01f4833223c21d7
I20260808 22:32:09.965579 12555 ts_tablet_manager.cc:1918] T b660396654e1494bb01f4833223c21d7 P 35c0c3409b0145eba8283e58d3080673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 22:32:09.965646 12555 log.cc:1196] T b660396654e1494bb01f4833223c21d7 P 35c0c3409b0145eba8283e58d3080673: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-1/wal/wals/b660396654e1494bb01f4833223c21d7
I20260808 22:32:09.965840 12557 ts_tablet_manager.cc:1939] T b660396654e1494bb01f4833223c21d7 P c2b3097001874d8ab050b0b6f8823a76: Deleting consensus metadata
I20260808 22:32:09.965891 12555 ts_tablet_manager.cc:1939] T b660396654e1494bb01f4833223c21d7 P 35c0c3409b0145eba8283e58d3080673: Deleting consensus metadata
I20260808 22:32:09.966456 11654 catalog_manager.cc:4841] TS c2b3097001874d8ab050b0b6f8823a76 (127.31.87.129:40109): tablet b660396654e1494bb01f4833223c21d7 (table test-table [id=e8a4f83121d54f91a14f6bcee7b4630a]) successfully deleted
I20260808 22:32:09.966557 11650 catalog_manager.cc:4841] TS 35c0c3409b0145eba8283e58d3080673 (127.31.87.130:41999): tablet b660396654e1494bb01f4833223c21d7 (table test-table [id=e8a4f83121d54f91a14f6bcee7b4630a]) successfully deleted
I20260808 22:32:09.967128 12556 ts_tablet_manager.cc:1905] T b660396654e1494bb01f4833223c21d7 P a8382975b0ef4c78a8aa87bb78abda54: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:32:09.969465 12556 ts_tablet_manager.cc:1918] T b660396654e1494bb01f4833223c21d7 P a8382975b0ef4c78a8aa87bb78abda54: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 22:32:09.969551 12556 log.cc:1196] T b660396654e1494bb01f4833223c21d7 P a8382975b0ef4c78a8aa87bb78abda54: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-2/wal/wals/b660396654e1494bb01f4833223c21d7
I20260808 22:32:09.969816 12556 ts_tablet_manager.cc:1939] T b660396654e1494bb01f4833223c21d7 P a8382975b0ef4c78a8aa87bb78abda54: Deleting consensus metadata
I20260808 22:32:09.970433 11650 catalog_manager.cc:4841] TS a8382975b0ef4c78a8aa87bb78abda54 (127.31.87.131:36531): tablet b660396654e1494bb01f4833223c21d7 (table test-table [id=e8a4f83121d54f91a14f6bcee7b4630a]) successfully deleted
Aug 08 22:32:09 dist-test-slave-266d krb5kdc[11600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228329, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 22:32:09 dist-test-slave-266d krb5kdc[11600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:32:09 dist-test-slave-266d krb5kdc[11600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 22:32:09.990088 11748 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:09.983676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36912 (local address 127.31.87.190:45745)
0808 22:32:09.983810 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:09.983814 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:09.983926 (+ 112us) server_negotiation.cc:407] Connection header received
0808 22:32:09.984029 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:09.984032 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:09.984089 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:09.984194 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:09.984865 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.985720 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.986608 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.986945 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.987055 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:09.989962 (+ 2907us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36912: BlockingRecv error: recv got EOF from 127.0.0.1:36912 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 08 22:32:09 dist-test-slave-266d krb5kdc[11600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228329, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM
I20260808 22:32:09.999233 11748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:09.991346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36920 (local address 127.31.87.190:45745)
0808 22:32:09.991449 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:09.991453 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:09.991627 (+ 174us) server_negotiation.cc:407] Connection header received
0808 22:32:09.991699 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:09.991702 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:09.991745 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:09.991829 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:09.992309 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.993046 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.993872 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:09.994213 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:09.994319 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:09.996564 (+ 2245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:09.996568 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:09.996570 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:09.996591 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:09.997910 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:09.998344 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:09.998346 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:09.998347 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:09.998408 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:09.998621 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:09.998622 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:09.998623 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:09.998766 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:09.998856 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:09.999041 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 22:32:09.999121 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 22:32:10.001678 11670 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36920:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:32:10.002140 11670 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:32:10.006982 12120 tablet_service.cc:1467] Processing CreateTablet for tablet bd043804407247d7a30757e934584c72 (DEFAULT_TABLE table=test-table [id=a882e2512a254b55bca194c098cf04b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:32:10.006981 11900 tablet_service.cc:1467] Processing CreateTablet for tablet bd043804407247d7a30757e934584c72 (DEFAULT_TABLE table=test-table [id=a882e2512a254b55bca194c098cf04b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:32:10.007279 11900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd043804407247d7a30757e934584c72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:32:10.007287 12120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd043804407247d7a30757e934584c72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:32:10.009160 12521 tablet_bootstrap.cc:492] T bd043804407247d7a30757e934584c72 P c2b3097001874d8ab050b0b6f8823a76: Bootstrap starting.
I20260808 22:32:10.009366 12360 tablet_service.cc:1467] Processing CreateTablet for tablet bd043804407247d7a30757e934584c72 (DEFAULT_TABLE table=test-table [id=a882e2512a254b55bca194c098cf04b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:32:10.009632 12360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd043804407247d7a30757e934584c72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:32:10.009964 12521 tablet_bootstrap.cc:654] T bd043804407247d7a30757e934584c72 P c2b3097001874d8ab050b0b6f8823a76: Neither blocks nor log segments found. Creating new log.
I20260808 22:32:10.011016 12521 tablet_bootstrap.cc:492] T bd043804407247d7a30757e934584c72 P c2b3097001874d8ab050b0b6f8823a76: No bootstrap required, opened a new log
I20260808 22:32:10.011094 12521 ts_tablet_manager.cc:1397] T bd043804407247d7a30757e934584c72 P c2b3097001874d8ab050b0b6f8823a76: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 22:32:10.011485 12521 raft_consensus.cc:348] T bd043804407247d7a30757e934584c72 P c2b3097001874d8ab050b0b6f8823a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } } peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } }
I20260808 22:32:10.011576 12521 raft_consensus.cc:374] T bd043804407247d7a30757e934584c72 P c2b3097001874d8ab050b0b6f8823a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:32:10.011606 12521 raft_consensus.cc:729] T bd043804407247d7a30757e934584c72 P c2b3097001874d8ab050b0b6f8823a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2b3097001874d8ab050b0b6f8823a76, State: Initialized, Role: FOLLOWER
I20260808 22:32:10.011720 12521 consensus_queue.cc:260] T bd043804407247d7a30757e934584c72 P c2b3097001874d8ab050b0b6f8823a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } } peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } }
I20260808 22:32:10.011960 12521 ts_tablet_manager.cc:1428] T bd043804407247d7a30757e934584c72 P c2b3097001874d8ab050b0b6f8823a76: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 22:32:10.011987 12523 tablet_bootstrap.cc:492] T bd043804407247d7a30757e934584c72 P a8382975b0ef4c78a8aa87bb78abda54: Bootstrap starting.
I20260808 22:32:10.012846 12523 tablet_bootstrap.cc:654] T bd043804407247d7a30757e934584c72 P a8382975b0ef4c78a8aa87bb78abda54: Neither blocks nor log segments found. Creating new log.
I20260808 22:32:10.014055 12523 tablet_bootstrap.cc:492] T bd043804407247d7a30757e934584c72 P a8382975b0ef4c78a8aa87bb78abda54: No bootstrap required, opened a new log
I20260808 22:32:10.014158 12523 ts_tablet_manager.cc:1397] T bd043804407247d7a30757e934584c72 P a8382975b0ef4c78a8aa87bb78abda54: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 22:32:10.014523 12523 raft_consensus.cc:348] T bd043804407247d7a30757e934584c72 P a8382975b0ef4c78a8aa87bb78abda54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } } peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } }
I20260808 22:32:10.014621 12523 raft_consensus.cc:374] T bd043804407247d7a30757e934584c72 P a8382975b0ef4c78a8aa87bb78abda54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:32:10.014649 12523 raft_consensus.cc:729] T bd043804407247d7a30757e934584c72 P a8382975b0ef4c78a8aa87bb78abda54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8382975b0ef4c78a8aa87bb78abda54, State: Initialized, Role: FOLLOWER
I20260808 22:32:10.014757 12523 consensus_queue.cc:260] T bd043804407247d7a30757e934584c72 P a8382975b0ef4c78a8aa87bb78abda54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } } peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } }
I20260808 22:32:10.014983 12523 ts_tablet_manager.cc:1428] T bd043804407247d7a30757e934584c72 P a8382975b0ef4c78a8aa87bb78abda54: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 22:32:10.015196 12519 tablet_bootstrap.cc:492] T bd043804407247d7a30757e934584c72 P 35c0c3409b0145eba8283e58d3080673: Bootstrap starting.
I20260808 22:32:10.015970 12519 tablet_bootstrap.cc:654] T bd043804407247d7a30757e934584c72 P 35c0c3409b0145eba8283e58d3080673: Neither blocks nor log segments found. Creating new log.
I20260808 22:32:10.017153 12519 tablet_bootstrap.cc:492] T bd043804407247d7a30757e934584c72 P 35c0c3409b0145eba8283e58d3080673: No bootstrap required, opened a new log
I20260808 22:32:10.017233 12519 ts_tablet_manager.cc:1397] T bd043804407247d7a30757e934584c72 P 35c0c3409b0145eba8283e58d3080673: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 22:32:10.017571 12519 raft_consensus.cc:348] T bd043804407247d7a30757e934584c72 P 35c0c3409b0145eba8283e58d3080673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } } peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } }
I20260808 22:32:10.017674 12519 raft_consensus.cc:374] T bd043804407247d7a30757e934584c72 P 35c0c3409b0145eba8283e58d3080673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:32:10.017701 12519 raft_consensus.cc:729] T bd043804407247d7a30757e934584c72 P 35c0c3409b0145eba8283e58d3080673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35c0c3409b0145eba8283e58d3080673, State: Initialized, Role: FOLLOWER
I20260808 22:32:10.017810 12519 consensus_queue.cc:260] T bd043804407247d7a30757e934584c72 P 35c0c3409b0145eba8283e58d3080673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } } peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } }
I20260808 22:32:10.017899 12593 raft_consensus.cc:482] T bd043804407247d7a30757e934584c72 P 35c0c3409b0145eba8283e58d3080673 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:32:10.018031 12519 ts_tablet_manager.cc:1428] T bd043804407247d7a30757e934584c72 P 35c0c3409b0145eba8283e58d3080673: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 22:32:10.017988 12593 raft_consensus.cc:504] T bd043804407247d7a30757e934584c72 P 35c0c3409b0145eba8283e58d3080673 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } } peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } }
I20260808 22:32:10.018390 12593 leader_election.cc:288] T bd043804407247d7a30757e934584c72 P 35c0c3409b0145eba8283e58d3080673 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2b3097001874d8ab050b0b6f8823a76 (127.31.87.129:40109), a8382975b0ef4c78a8aa87bb78abda54 (127.31.87.131:36531)
I20260808 22:32:10.018894 12395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd043804407247d7a30757e934584c72" candidate_uuid: "35c0c3409b0145eba8283e58d3080673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8382975b0ef4c78a8aa87bb78abda54" is_pre_election: true
I20260808 22:32:10.019013 12395 raft_consensus.cc:2454] T bd043804407247d7a30757e934584c72 P a8382975b0ef4c78a8aa87bb78abda54 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35c0c3409b0145eba8283e58d3080673 in term 0.
I20260808 22:32:10.019021 11932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd043804407247d7a30757e934584c72" candidate_uuid: "35c0c3409b0145eba8283e58d3080673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2b3097001874d8ab050b0b6f8823a76" is_pre_election: true
I20260808 22:32:10.019150 11932 raft_consensus.cc:2454] T bd043804407247d7a30757e934584c72 P c2b3097001874d8ab050b0b6f8823a76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35c0c3409b0145eba8283e58d3080673 in term 0.
I20260808 22:32:10.019307 12040 leader_election.cc:302] T bd043804407247d7a30757e934584c72 P 35c0c3409b0145eba8283e58d3080673 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35c0c3409b0145eba8283e58d3080673, a8382975b0ef4c78a8aa87bb78abda54; no voters:
I20260808 22:32:10.019471 12593 raft_consensus.cc:2781] T bd043804407247d7a30757e934584c72 P 35c0c3409b0145eba8283e58d3080673 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:32:10.019524 12593 raft_consensus.cc:482] T bd043804407247d7a30757e934584c72 P 35c0c3409b0145eba8283e58d3080673 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:32:10.019567 12593 raft_consensus.cc:3037] T bd043804407247d7a30757e934584c72 P 35c0c3409b0145eba8283e58d3080673 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:32:10.020586 12593 raft_consensus.cc:504] T bd043804407247d7a30757e934584c72 P 35c0c3409b0145eba8283e58d3080673 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } } peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } }
I20260808 22:32:10.020888 12593 leader_election.cc:288] T bd043804407247d7a30757e934584c72 P 35c0c3409b0145eba8283e58d3080673 [CANDIDATE]: Term 1 election: Requested vote from peers c2b3097001874d8ab050b0b6f8823a76 (127.31.87.129:40109), a8382975b0ef4c78a8aa87bb78abda54 (127.31.87.131:36531)
I20260808 22:32:10.021190 12395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd043804407247d7a30757e934584c72" candidate_uuid: "35c0c3409b0145eba8283e58d3080673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8382975b0ef4c78a8aa87bb78abda54"
I20260808 22:32:10.021200 11932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd043804407247d7a30757e934584c72" candidate_uuid: "35c0c3409b0145eba8283e58d3080673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2b3097001874d8ab050b0b6f8823a76"
I20260808 22:32:10.021294 12395 raft_consensus.cc:3037] T bd043804407247d7a30757e934584c72 P a8382975b0ef4c78a8aa87bb78abda54 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:32:10.021298 11932 raft_consensus.cc:3037] T bd043804407247d7a30757e934584c72 P c2b3097001874d8ab050b0b6f8823a76 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:32:10.022090 12395 raft_consensus.cc:2454] T bd043804407247d7a30757e934584c72 P a8382975b0ef4c78a8aa87bb78abda54 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35c0c3409b0145eba8283e58d3080673 in term 1.
I20260808 22:32:10.022332 12040 leader_election.cc:302] T bd043804407247d7a30757e934584c72 P 35c0c3409b0145eba8283e58d3080673 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35c0c3409b0145eba8283e58d3080673, a8382975b0ef4c78a8aa87bb78abda54; no voters:
I20260808 22:32:10.022365 11932 raft_consensus.cc:2454] T bd043804407247d7a30757e934584c72 P c2b3097001874d8ab050b0b6f8823a76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35c0c3409b0145eba8283e58d3080673 in term 1.
I20260808 22:32:10.022497 12593 raft_consensus.cc:2781] T bd043804407247d7a30757e934584c72 P 35c0c3409b0145eba8283e58d3080673 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:32:10.022609 12593 raft_consensus.cc:686] T bd043804407247d7a30757e934584c72 P 35c0c3409b0145eba8283e58d3080673 [term 1 LEADER]: Becoming Leader. State: Replica: 35c0c3409b0145eba8283e58d3080673, State: Running, Role: LEADER
I20260808 22:32:10.022743 12593 consensus_queue.cc:237] T bd043804407247d7a30757e934584c72 P 35c0c3409b0145eba8283e58d3080673 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } } peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } }
I20260808 22:32:10.024065 11671 catalog_manager.cc:5495] T bd043804407247d7a30757e934584c72 P 35c0c3409b0145eba8283e58d3080673 reported cstate change: term changed from 0 to 1, leader changed from <none> to 35c0c3409b0145eba8283e58d3080673 (127.31.87.130). New cstate: current_term: 1 leader_uuid: "35c0c3409b0145eba8283e58d3080673" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } health_report { overall_health: UNKNOWN } } }
I20260808 22:32:10.033149 12512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:10.029980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43442 (local address 127.31.87.130:41999)
0808 22:32:10.030087 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:10.030091 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:10.030103 (+ 12us) server_negotiation.cc:407] Connection header received
0808 22:32:10.030224 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:10.030226 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:10.030265 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:10.030322 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:32:10.030572 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:10.031056 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:10.032225 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:10.032419 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:10.032482 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:10.032768 (+ 286us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:32:10.032830 (+ 62us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:32:10.032873 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:10.032971 (+ 98us) server_negotiation.cc:299] Negotiation successful
0808 22:32:10.033012 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 22:32:10.034827 11932 raft_consensus.cc:1261] T bd043804407247d7a30757e934584c72 P c2b3097001874d8ab050b0b6f8823a76 [term 1 FOLLOWER]: Refusing update from remote peer 35c0c3409b0145eba8283e58d3080673: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:32:10.034870 12395 raft_consensus.cc:1261] T bd043804407247d7a30757e934584c72 P a8382975b0ef4c78a8aa87bb78abda54 [term 1 FOLLOWER]: Refusing update from remote peer 35c0c3409b0145eba8283e58d3080673: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:32:10.035234 12530 consensus_queue.cc:1035] T bd043804407247d7a30757e934584c72 P 35c0c3409b0145eba8283e58d3080673 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:32:10.035394 12593 consensus_queue.cc:1035] T bd043804407247d7a30757e934584c72 P 35c0c3409b0145eba8283e58d3080673 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:32:10.041348 11671 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36920:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:32:10.041477 11671 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36920:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:32:10.044221 11671 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9f31f95f36104c6fa687563b1fdb4d24: Sending DeleteTablet for 3 replicas of tablet bd043804407247d7a30757e934584c72
I20260808 22:32:10.044863 12120 tablet_service.cc:1514] Processing DeleteTablet for tablet bd043804407247d7a30757e934584c72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:32:10 UTC) from {username='oryx', principal='oryx/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:34662
I20260808 22:32:10.044878 11900 tablet_service.cc:1514] Processing DeleteTablet for tablet bd043804407247d7a30757e934584c72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:32:10 UTC) from {username='oryx', principal='oryx/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:57510
I20260808 22:32:10.045035 12555 tablet_replica.cc:330] T bd043804407247d7a30757e934584c72 P 35c0c3409b0145eba8283e58d3080673: stopping tablet replica
I20260808 22:32:10.045179 12557 tablet_replica.cc:330] T bd043804407247d7a30757e934584c72 P c2b3097001874d8ab050b0b6f8823a76: stopping tablet replica
I20260808 22:32:10.045161 12360 tablet_service.cc:1514] Processing DeleteTablet for tablet bd043804407247d7a30757e934584c72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:32:10 UTC) from {username='oryx', principal='oryx/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:38168
I20260808 22:32:10.045182 12555 raft_consensus.cc:2229] T bd043804407247d7a30757e934584c72 P 35c0c3409b0145eba8283e58d3080673 [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:32:10.045280 12557 raft_consensus.cc:2229] T bd043804407247d7a30757e934584c72 P c2b3097001874d8ab050b0b6f8823a76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:32:10.045311 12556 tablet_replica.cc:330] T bd043804407247d7a30757e934584c72 P a8382975b0ef4c78a8aa87bb78abda54: stopping tablet replica
I20260808 22:32:10.045336 12555 raft_consensus.cc:2258] T bd043804407247d7a30757e934584c72 P 35c0c3409b0145eba8283e58d3080673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:32:10.045389 12557 raft_consensus.cc:2258] T bd043804407247d7a30757e934584c72 P c2b3097001874d8ab050b0b6f8823a76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:32:10.045396 12556 raft_consensus.cc:2229] T bd043804407247d7a30757e934584c72 P a8382975b0ef4c78a8aa87bb78abda54 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:32:10.045491 12556 raft_consensus.cc:2258] T bd043804407247d7a30757e934584c72 P a8382975b0ef4c78a8aa87bb78abda54 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:32:10.045925 11671 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36920:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:32:10.046300 11671 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:32:10.046860 12557 ts_tablet_manager.cc:1905] T bd043804407247d7a30757e934584c72 P c2b3097001874d8ab050b0b6f8823a76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:32:10.046900 12555 ts_tablet_manager.cc:1905] T bd043804407247d7a30757e934584c72 P 35c0c3409b0145eba8283e58d3080673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:32:10.047231 12556 ts_tablet_manager.cc:1905] T bd043804407247d7a30757e934584c72 P a8382975b0ef4c78a8aa87bb78abda54: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:32:10.049154 12557 ts_tablet_manager.cc:1918] T bd043804407247d7a30757e934584c72 P c2b3097001874d8ab050b0b6f8823a76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:32:10.049196 12555 ts_tablet_manager.cc:1918] T bd043804407247d7a30757e934584c72 P 35c0c3409b0145eba8283e58d3080673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:32:10.049220 12557 log.cc:1196] T bd043804407247d7a30757e934584c72 P c2b3097001874d8ab050b0b6f8823a76: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-0/wal/wals/bd043804407247d7a30757e934584c72
I20260808 22:32:10.049264 12555 log.cc:1196] T bd043804407247d7a30757e934584c72 P 35c0c3409b0145eba8283e58d3080673: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-1/wal/wals/bd043804407247d7a30757e934584c72
I20260808 22:32:10.049463 12557 ts_tablet_manager.cc:1939] T bd043804407247d7a30757e934584c72 P c2b3097001874d8ab050b0b6f8823a76: Deleting consensus metadata
I20260808 22:32:10.049497 12555 ts_tablet_manager.cc:1939] T bd043804407247d7a30757e934584c72 P 35c0c3409b0145eba8283e58d3080673: Deleting consensus metadata
I20260808 22:32:10.050328 12556 ts_tablet_manager.cc:1918] T bd043804407247d7a30757e934584c72 P a8382975b0ef4c78a8aa87bb78abda54: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:32:10.050403 12556 log.cc:1196] T bd043804407247d7a30757e934584c72 P a8382975b0ef4c78a8aa87bb78abda54: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-2/wal/wals/bd043804407247d7a30757e934584c72
I20260808 22:32:10.050405 11654 catalog_manager.cc:4841] TS c2b3097001874d8ab050b0b6f8823a76 (127.31.87.129:40109): tablet bd043804407247d7a30757e934584c72 (table test-table [id=a882e2512a254b55bca194c098cf04b9]) successfully deleted
I20260808 22:32:10.050532 11650 catalog_manager.cc:4841] TS 35c0c3409b0145eba8283e58d3080673 (127.31.87.130:41999): tablet bd043804407247d7a30757e934584c72 (table test-table [id=a882e2512a254b55bca194c098cf04b9]) successfully deleted
I20260808 22:32:10.050640 12556 ts_tablet_manager.cc:1939] T bd043804407247d7a30757e934584c72 P a8382975b0ef4c78a8aa87bb78abda54: Deleting consensus metadata
I20260808 22:32:10.051192 11650 catalog_manager.cc:4841] TS a8382975b0ef4c78a8aa87bb78abda54 (127.31.87.131:36531): tablet bd043804407247d7a30757e934584c72 (table test-table [id=a882e2512a254b55bca194c098cf04b9]) successfully deleted
I20260808 22:32:10.053136 11900 tablet_service.cc:1467] Processing CreateTablet for tablet e6641b87347b47a8ae07c3d0336652a6 (DEFAULT_TABLE table=test-table [id=d464b19901664059a205094a78adf453]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:32:10.053287 12120 tablet_service.cc:1467] Processing CreateTablet for tablet e6641b87347b47a8ae07c3d0336652a6 (DEFAULT_TABLE table=test-table [id=d464b19901664059a205094a78adf453]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:32:10.053382 11900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6641b87347b47a8ae07c3d0336652a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:32:10.053372 12360 tablet_service.cc:1467] Processing CreateTablet for tablet e6641b87347b47a8ae07c3d0336652a6 (DEFAULT_TABLE table=test-table [id=d464b19901664059a205094a78adf453]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:32:10.053503 12120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6641b87347b47a8ae07c3d0336652a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:32:10.053577 12360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6641b87347b47a8ae07c3d0336652a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:32:10.055460 12521 tablet_bootstrap.cc:492] T e6641b87347b47a8ae07c3d0336652a6 P c2b3097001874d8ab050b0b6f8823a76: Bootstrap starting.
I20260808 22:32:10.055509 12523 tablet_bootstrap.cc:492] T e6641b87347b47a8ae07c3d0336652a6 P a8382975b0ef4c78a8aa87bb78abda54: Bootstrap starting.
I20260808 22:32:10.055576 12519 tablet_bootstrap.cc:492] T e6641b87347b47a8ae07c3d0336652a6 P 35c0c3409b0145eba8283e58d3080673: Bootstrap starting.
I20260808 22:32:10.056272 12523 tablet_bootstrap.cc:654] T e6641b87347b47a8ae07c3d0336652a6 P a8382975b0ef4c78a8aa87bb78abda54: Neither blocks nor log segments found. Creating new log.
I20260808 22:32:10.056298 12521 tablet_bootstrap.cc:654] T e6641b87347b47a8ae07c3d0336652a6 P c2b3097001874d8ab050b0b6f8823a76: Neither blocks nor log segments found. Creating new log.
I20260808 22:32:10.056351 12519 tablet_bootstrap.cc:654] T e6641b87347b47a8ae07c3d0336652a6 P 35c0c3409b0145eba8283e58d3080673: Neither blocks nor log segments found. Creating new log.
I20260808 22:32:10.057355 12521 tablet_bootstrap.cc:492] T e6641b87347b47a8ae07c3d0336652a6 P c2b3097001874d8ab050b0b6f8823a76: No bootstrap required, opened a new log
I20260808 22:32:10.057440 12521 ts_tablet_manager.cc:1397] T e6641b87347b47a8ae07c3d0336652a6 P c2b3097001874d8ab050b0b6f8823a76: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 22:32:10.057766 12519 tablet_bootstrap.cc:492] T e6641b87347b47a8ae07c3d0336652a6 P 35c0c3409b0145eba8283e58d3080673: No bootstrap required, opened a new log
I20260808 22:32:10.057835 12519 ts_tablet_manager.cc:1397] T e6641b87347b47a8ae07c3d0336652a6 P 35c0c3409b0145eba8283e58d3080673: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 22:32:10.057847 12521 raft_consensus.cc:348] T e6641b87347b47a8ae07c3d0336652a6 P c2b3097001874d8ab050b0b6f8823a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } } peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } }
I20260808 22:32:10.057945 12521 raft_consensus.cc:374] T e6641b87347b47a8ae07c3d0336652a6 P c2b3097001874d8ab050b0b6f8823a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:32:10.057976 12521 raft_consensus.cc:729] T e6641b87347b47a8ae07c3d0336652a6 P c2b3097001874d8ab050b0b6f8823a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2b3097001874d8ab050b0b6f8823a76, State: Initialized, Role: FOLLOWER
I20260808 22:32:10.058084 12521 consensus_queue.cc:260] T e6641b87347b47a8ae07c3d0336652a6 P c2b3097001874d8ab050b0b6f8823a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } } peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } }
I20260808 22:32:10.058254 12523 tablet_bootstrap.cc:492] T e6641b87347b47a8ae07c3d0336652a6 P a8382975b0ef4c78a8aa87bb78abda54: No bootstrap required, opened a new log
I20260808 22:32:10.058202 12519 raft_consensus.cc:348] T e6641b87347b47a8ae07c3d0336652a6 P 35c0c3409b0145eba8283e58d3080673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } } peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } }
I20260808 22:32:10.058291 12519 raft_consensus.cc:374] T e6641b87347b47a8ae07c3d0336652a6 P 35c0c3409b0145eba8283e58d3080673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:32:10.058320 12523 ts_tablet_manager.cc:1397] T e6641b87347b47a8ae07c3d0336652a6 P a8382975b0ef4c78a8aa87bb78abda54: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260808 22:32:10.058319 12519 raft_consensus.cc:729] T e6641b87347b47a8ae07c3d0336652a6 P 35c0c3409b0145eba8283e58d3080673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35c0c3409b0145eba8283e58d3080673, State: Initialized, Role: FOLLOWER
I20260808 22:32:10.058341 12521 ts_tablet_manager.cc:1428] T e6641b87347b47a8ae07c3d0336652a6 P c2b3097001874d8ab050b0b6f8823a76: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 22:32:10.058420 12519 consensus_queue.cc:260] T e6641b87347b47a8ae07c3d0336652a6 P 35c0c3409b0145eba8283e58d3080673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } } peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } }
I20260808 22:32:10.058641 12519 ts_tablet_manager.cc:1428] T e6641b87347b47a8ae07c3d0336652a6 P 35c0c3409b0145eba8283e58d3080673: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 22:32:10.058663 12523 raft_consensus.cc:348] T e6641b87347b47a8ae07c3d0336652a6 P a8382975b0ef4c78a8aa87bb78abda54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } } peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } }
I20260808 22:32:10.058758 12523 raft_consensus.cc:374] T e6641b87347b47a8ae07c3d0336652a6 P a8382975b0ef4c78a8aa87bb78abda54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:32:10.058804 12523 raft_consensus.cc:729] T e6641b87347b47a8ae07c3d0336652a6 P a8382975b0ef4c78a8aa87bb78abda54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8382975b0ef4c78a8aa87bb78abda54, State: Initialized, Role: FOLLOWER
I20260808 22:32:10.058909 12523 consensus_queue.cc:260] T e6641b87347b47a8ae07c3d0336652a6 P a8382975b0ef4c78a8aa87bb78abda54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } } peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } }
I20260808 22:32:10.059136 12523 ts_tablet_manager.cc:1428] T e6641b87347b47a8ae07c3d0336652a6 P a8382975b0ef4c78a8aa87bb78abda54: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 22:32:10.195175 12534 raft_consensus.cc:482] T e6641b87347b47a8ae07c3d0336652a6 P c2b3097001874d8ab050b0b6f8823a76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:32:10.195322 12534 raft_consensus.cc:504] T e6641b87347b47a8ae07c3d0336652a6 P c2b3097001874d8ab050b0b6f8823a76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } } peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } }
I20260808 22:32:10.196399 12534 leader_election.cc:288] T e6641b87347b47a8ae07c3d0336652a6 P c2b3097001874d8ab050b0b6f8823a76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35c0c3409b0145eba8283e58d3080673 (127.31.87.130:41999), a8382975b0ef4c78a8aa87bb78abda54 (127.31.87.131:36531)
I20260808 22:32:10.200325 12512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:10.196693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:54941 (local address 127.31.87.130:41999)
0808 22:32:10.196825 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:10.196829 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:10.196946 (+ 117us) server_negotiation.cc:407] Connection header received
0808 22:32:10.197106 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:10.197109 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:10.197166 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:10.197241 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:32:10.197548 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:10.198169 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:10.199313 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:10.199735 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:10.199835 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:10.199892 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:10.200148 (+ 256us) server_negotiation.cc:299] Negotiation successful
0808 22:32:10.200212 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 22:32:10.200326 12518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:10.197044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:42519 (local address 127.31.87.131:36531)
0808 22:32:10.197192 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:10.197196 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:10.197404 (+ 208us) server_negotiation.cc:407] Connection header received
0808 22:32:10.197499 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:10.197503 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:10.197550 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:10.197605 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:32:10.197859 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:10.198385 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:10.199381 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:10.199749 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:10.199851 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:10.199901 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:10.200141 (+ 240us) server_negotiation.cc:299] Negotiation successful
0808 22:32:10.200207 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 22:32:10.200350 12644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:10.196953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.131:36531 (local address 127.31.87.129:42519)
0808 22:32:10.197309 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:10.197322 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:10.197416 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:10.197631 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:10.197633 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:10.197650 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:32:10.197767 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:10.197773 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:10.198484 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:10.198487 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:10.199265 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:10.199276 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:10.199883 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:10.199887 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:10.200021 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:10.200048 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 22:32:10.200121 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 22:32:10.200192 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"mutex_wait_us":34,"thread_start_us":97,"threads_started":1}
I20260808 22:32:10.200353 12642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:10.196599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.130:41999 (local address 127.31.87.129:54941)
0808 22:32:10.196861 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:10.196876 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:10.196983 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:10.197269 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:10.197272 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:10.197292 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:32:10.197431 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:10.197437 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:10.198294 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:10.198298 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:10.199188 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:10.199197 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:10.199880 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:10.199884 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:10.200010 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:10.200046 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 22:32:10.200111 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 22:32:10.200181 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":51,"thread_start_us":70,"threads_started":1}
I20260808 22:32:10.200881 12395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6641b87347b47a8ae07c3d0336652a6" candidate_uuid: "c2b3097001874d8ab050b0b6f8823a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8382975b0ef4c78a8aa87bb78abda54" is_pre_election: true
I20260808 22:32:10.200932 12151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6641b87347b47a8ae07c3d0336652a6" candidate_uuid: "c2b3097001874d8ab050b0b6f8823a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35c0c3409b0145eba8283e58d3080673" is_pre_election: true
I20260808 22:32:10.201010 12395 raft_consensus.cc:2454] T e6641b87347b47a8ae07c3d0336652a6 P a8382975b0ef4c78a8aa87bb78abda54 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2b3097001874d8ab050b0b6f8823a76 in term 0.
I20260808 22:32:10.201059 12151 raft_consensus.cc:2454] T e6641b87347b47a8ae07c3d0336652a6 P 35c0c3409b0145eba8283e58d3080673 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2b3097001874d8ab050b0b6f8823a76 in term 0.
I20260808 22:32:10.201395 11815 leader_election.cc:302] T e6641b87347b47a8ae07c3d0336652a6 P c2b3097001874d8ab050b0b6f8823a76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8382975b0ef4c78a8aa87bb78abda54, c2b3097001874d8ab050b0b6f8823a76; no voters:
I20260808 22:32:10.201614 12534 raft_consensus.cc:2781] T e6641b87347b47a8ae07c3d0336652a6 P c2b3097001874d8ab050b0b6f8823a76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:32:10.201668 12534 raft_consensus.cc:482] T e6641b87347b47a8ae07c3d0336652a6 P c2b3097001874d8ab050b0b6f8823a76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:32:10.201692 12534 raft_consensus.cc:3037] T e6641b87347b47a8ae07c3d0336652a6 P c2b3097001874d8ab050b0b6f8823a76 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:32:10.202595 12534 raft_consensus.cc:504] T e6641b87347b47a8ae07c3d0336652a6 P c2b3097001874d8ab050b0b6f8823a76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } } peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } }
I20260808 22:32:10.202899 12534 leader_election.cc:288] T e6641b87347b47a8ae07c3d0336652a6 P c2b3097001874d8ab050b0b6f8823a76 [CANDIDATE]: Term 1 election: Requested vote from peers 35c0c3409b0145eba8283e58d3080673 (127.31.87.130:41999), a8382975b0ef4c78a8aa87bb78abda54 (127.31.87.131:36531)
I20260808 22:32:10.203193 12151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6641b87347b47a8ae07c3d0336652a6" candidate_uuid: "c2b3097001874d8ab050b0b6f8823a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35c0c3409b0145eba8283e58d3080673"
I20260808 22:32:10.203269 12395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6641b87347b47a8ae07c3d0336652a6" candidate_uuid: "c2b3097001874d8ab050b0b6f8823a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8382975b0ef4c78a8aa87bb78abda54"
I20260808 22:32:10.203315 12151 raft_consensus.cc:3037] T e6641b87347b47a8ae07c3d0336652a6 P 35c0c3409b0145eba8283e58d3080673 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:32:10.203375 12395 raft_consensus.cc:3037] T e6641b87347b47a8ae07c3d0336652a6 P a8382975b0ef4c78a8aa87bb78abda54 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:32:10.204190 12395 raft_consensus.cc:2454] T e6641b87347b47a8ae07c3d0336652a6 P a8382975b0ef4c78a8aa87bb78abda54 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2b3097001874d8ab050b0b6f8823a76 in term 1.
I20260808 22:32:10.204192 12151 raft_consensus.cc:2454] T e6641b87347b47a8ae07c3d0336652a6 P 35c0c3409b0145eba8283e58d3080673 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2b3097001874d8ab050b0b6f8823a76 in term 1.
I20260808 22:32:10.204438 11815 leader_election.cc:302] T e6641b87347b47a8ae07c3d0336652a6 P c2b3097001874d8ab050b0b6f8823a76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35c0c3409b0145eba8283e58d3080673, c2b3097001874d8ab050b0b6f8823a76; no voters:
I20260808 22:32:10.204595 12534 raft_consensus.cc:2781] T e6641b87347b47a8ae07c3d0336652a6 P c2b3097001874d8ab050b0b6f8823a76 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:32:10.204797 12534 raft_consensus.cc:686] T e6641b87347b47a8ae07c3d0336652a6 P c2b3097001874d8ab050b0b6f8823a76 [term 1 LEADER]: Becoming Leader. State: Replica: c2b3097001874d8ab050b0b6f8823a76, State: Running, Role: LEADER
I20260808 22:32:10.205029 12534 consensus_queue.cc:237] T e6641b87347b47a8ae07c3d0336652a6 P c2b3097001874d8ab050b0b6f8823a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } } peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } }
I20260808 22:32:10.207253 11672 catalog_manager.cc:5495] T e6641b87347b47a8ae07c3d0336652a6 P c2b3097001874d8ab050b0b6f8823a76 reported cstate change: term changed from 0 to 1, leader changed from <none> to c2b3097001874d8ab050b0b6f8823a76 (127.31.87.129). New cstate: current_term: 1 leader_uuid: "c2b3097001874d8ab050b0b6f8823a76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2b3097001874d8ab050b0b6f8823a76" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40109 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 } health_report { overall_health: UNKNOWN } } }
I20260808 22:32:10.254156 12650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:10.250905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38176 (local address 127.31.87.129:40109)
0808 22:32:10.251160 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:10.251164 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:10.251174 (+ 10us) server_negotiation.cc:407] Connection header received
0808 22:32:10.251245 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:10.251247 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:10.251295 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:10.251391 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:32:10.251564 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:10.252080 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:10.252817 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:10.253027 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:10.253115 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:10.253379 (+ 264us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:32:10.253823 (+ 444us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:32:10.253904 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:10.253949 (+ 45us) server_negotiation.cc:299] Negotiation successful
0808 22:32:10.253994 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":119,"threads_started":1}
I20260808 22:32:10.263454 12512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:10.260381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:47757 (local address 127.31.87.130:41999)
0808 22:32:10.260497 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:10.260500 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:10.260557 (+ 57us) server_negotiation.cc:407] Connection header received
0808 22:32:10.260701 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:10.260704 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:10.260746 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:10.260804 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:32:10.261030 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:10.261578 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:10.262484 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:10.262698 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:10.262781 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:10.263057 (+ 276us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:32:10.263130 (+ 73us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:32:10.263185 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:10.263278 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 22:32:10.263326 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 22:32:10.263476 12658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:10.260205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.130:41999 (local address 127.31.87.129:47757)
0808 22:32:10.260480 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:10.260495 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:10.260596 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:10.260825 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:10.260828 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:10.260835 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 22:32:10.260937 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:10.260942 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:10.261678 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:10.261682 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:10.262380 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:10.262389 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:10.262791 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:10.262795 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:10.262884 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:10.262922 (+ 38us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 22:32:10.263205 (+ 283us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 22:32:10.263212 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 22:32:10.263268 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 22:32:10.263346 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":88,"threads_started":1}
I20260808 22:32:10.268698 12151 raft_consensus.cc:1261] T e6641b87347b47a8ae07c3d0336652a6 P 35c0c3409b0145eba8283e58d3080673 [term 1 FOLLOWER]: Refusing update from remote peer c2b3097001874d8ab050b0b6f8823a76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:32:10.268698 12395 raft_consensus.cc:1261] T e6641b87347b47a8ae07c3d0336652a6 P a8382975b0ef4c78a8aa87bb78abda54 [term 1 FOLLOWER]: Refusing update from remote peer c2b3097001874d8ab050b0b6f8823a76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:32:10.269157 12647 consensus_queue.cc:1035] T e6641b87347b47a8ae07c3d0336652a6 P c2b3097001874d8ab050b0b6f8823a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35c0c3409b0145eba8283e58d3080673" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:32:10.269335 12534 consensus_queue.cc:1035] T e6641b87347b47a8ae07c3d0336652a6 P c2b3097001874d8ab050b0b6f8823a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8382975b0ef4c78a8aa87bb78abda54" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:32:10.284900 12596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:10.281596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:40109 (local address 127.31.87.130:36029)
0808 22:32:10.281753 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:10.281766 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:10.281835 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:10.282049 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:10.282052 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:10.282059 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 22:32:10.282146 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:10.282151 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:10.282872 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:10.282876 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:10.283442 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:10.283449 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:10.283951 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:10.283954 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:10.284013 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:10.284037 (+ 24us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 22:32:10.284618 (+ 581us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 22:32:10.284625 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 22:32:10.284692 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 22:32:10.284776 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260808 22:32:10.284900 12650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:10.281787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:36029 (local address 127.31.87.129:40109)
0808 22:32:10.281885 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:10.281889 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:10.281900 (+ 11us) server_negotiation.cc:407] Connection header received
0808 22:32:10.281934 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:10.281937 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:10.281975 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:10.282032 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:32:10.282234 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:10.282777 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:10.283545 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:10.283788 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:10.283883 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:10.284410 (+ 527us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:32:10.284517 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:32:10.284595 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:10.284711 (+ 116us) server_negotiation.cc:299] Negotiation successful
0808 22:32:10.284768 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 22:32:10.302531 11672 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36920:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:32:10.302711 11672 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36920:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:32:10.304703 11672 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9f31f95f36104c6fa687563b1fdb4d24: Sending DeleteTablet for 3 replicas of tablet e6641b87347b47a8ae07c3d0336652a6
I20260808 22:32:10.305300 12120 tablet_service.cc:1514] Processing DeleteTablet for tablet e6641b87347b47a8ae07c3d0336652a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:32:10 UTC) from {username='oryx', principal='oryx/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:34662
I20260808 22:32:10.305492 12555 tablet_replica.cc:330] T e6641b87347b47a8ae07c3d0336652a6 P 35c0c3409b0145eba8283e58d3080673: stopping tablet replica
I20260808 22:32:10.305505 12360 tablet_service.cc:1514] Processing DeleteTablet for tablet e6641b87347b47a8ae07c3d0336652a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:32:10 UTC) from {username='oryx', principal='oryx/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:38168
I20260808 22:32:10.305645 12555 raft_consensus.cc:2229] T e6641b87347b47a8ae07c3d0336652a6 P 35c0c3409b0145eba8283e58d3080673 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:32:10.305661 12556 tablet_replica.cc:330] T e6641b87347b47a8ae07c3d0336652a6 P a8382975b0ef4c78a8aa87bb78abda54: stopping tablet replica
I20260808 22:32:10.305760 12555 raft_consensus.cc:2258] T e6641b87347b47a8ae07c3d0336652a6 P 35c0c3409b0145eba8283e58d3080673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:32:10.305774 12556 raft_consensus.cc:2229] T e6641b87347b47a8ae07c3d0336652a6 P a8382975b0ef4c78a8aa87bb78abda54 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:32:10.305863 12556 raft_consensus.cc:2258] T e6641b87347b47a8ae07c3d0336652a6 P a8382975b0ef4c78a8aa87bb78abda54 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:32:10.306771 12556 ts_tablet_manager.cc:1905] T e6641b87347b47a8ae07c3d0336652a6 P a8382975b0ef4c78a8aa87bb78abda54: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:32:10.306785 12555 ts_tablet_manager.cc:1905] T e6641b87347b47a8ae07c3d0336652a6 P 35c0c3409b0145eba8283e58d3080673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:32:10.308722 12556 ts_tablet_manager.cc:1918] T e6641b87347b47a8ae07c3d0336652a6 P a8382975b0ef4c78a8aa87bb78abda54: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 22:32:10.308790 12556 log.cc:1196] T e6641b87347b47a8ae07c3d0336652a6 P a8382975b0ef4c78a8aa87bb78abda54: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-2/wal/wals/e6641b87347b47a8ae07c3d0336652a6
I20260808 22:32:10.309041 12556 ts_tablet_manager.cc:1939] T e6641b87347b47a8ae07c3d0336652a6 P a8382975b0ef4c78a8aa87bb78abda54: Deleting consensus metadata
I20260808 22:32:10.309018 11900 tablet_service.cc:1514] Processing DeleteTablet for tablet e6641b87347b47a8ae07c3d0336652a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:32:10 UTC) from {username='oryx', principal='oryx/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:57510
I20260808 22:32:10.309201 12557 tablet_replica.cc:330] T e6641b87347b47a8ae07c3d0336652a6 P c2b3097001874d8ab050b0b6f8823a76: stopping tablet replica
I20260808 22:32:10.309297 12557 raft_consensus.cc:2229] T e6641b87347b47a8ae07c3d0336652a6 P c2b3097001874d8ab050b0b6f8823a76 [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:32:10.309324 12555 ts_tablet_manager.cc:1918] T e6641b87347b47a8ae07c3d0336652a6 P 35c0c3409b0145eba8283e58d3080673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 22:32:10.309399 12555 log.cc:1196] T e6641b87347b47a8ae07c3d0336652a6 P 35c0c3409b0145eba8283e58d3080673: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786228291013016-32094-0/minicluster-data/ts-1/wal/wals/e6641b87347b47a8ae07c3d0336652a6
I20260808 22:32:10.309607 12557 raft_consensus.cc:2258] T e6641b87347b47a8ae07c3d0336652a6 P c2b3097001874d8ab050b0b6f8823a76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:32:10.309762 11650 catalog_manager.cc:4841] TS a8382975b0ef4c78a8aa87bb78abda54 (127.31.87.131:36531): tablet e6641b87347b47a8ae07c3d0336652a6 (table test-table [id=d464b19901664059a205094a78adf453]) successfully deleted
I20260808 22:32:10.309882 12555 ts_tablet_manager.cc:1939] T e6641b87347b47a8ae07c3d0336652a6 P 35c0c3409b0145eba8283e58d3080673: Deleting consensus metadata
I20260808 22:32:10.310137 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 11791
I20260808 22:32:10.320474 11650 catalog_manager.cc:4841] TS 35c0c3409b0145eba8283e58d3080673 (127.31.87.130:41999): tablet e6641b87347b47a8ae07c3d0336652a6 (table test-table [id=d464b19901664059a205094a78adf453]) successfully deleted
W20260808 22:32:10.326944 11654 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.87.129:40109 (error 108)
I20260808 22:32:10.327167 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 12013
I20260808 22:32:10.329542 12666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:10.329122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:40109 (local address 127.0.0.1:38184)
0808 22:32:10.329351 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:10.329434 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.129:40109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":59,"threads_started":1}
W20260808 22:32:10.329820 11654 catalog_manager.cc:4584] TS c2b3097001874d8ab050b0b6f8823a76 (127.31.87.129:40109): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e6641b87347b47a8ae07c3d0336652a6: Network error: Client connection negotiation failed: client connection to 127.31.87.129:40109: connect: Connection refused (error 111)
I20260808 22:32:10.338507 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 12251
I20260808 22:32:10.352593 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 11623
2026-08-08T22:32:10Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3169 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786228291013016-32094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:32:10 dist-test-slave-266d krb5kdc[12670](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:32:10 dist-test-slave-266d krb5kdc[12670](info): set up 2 sockets
Aug 08 22:32:10 dist-test-slave-266d krb5kdc[12670](info): commencing operation
krb5kdc: starting...
W20260808 22:32:11.505075 32094 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Aug 08 22:32:11 dist-test-slave-266d krb5kdc[12670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228331, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:32:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:32:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.87.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
I20260808 22:32:11.617537 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:38029
--webserver_interface=127.31.87.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:35645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:38029
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:32:11.726575 12691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:32:11.726809 12691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:32:11.726855 12691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:32:11.730733 12691 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:32:11.730783 12691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:32:11.730808 12691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:32:11.730844 12691 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:32:11.730875 12691 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:32:11.735939 12691 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:35645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786228291013016-32094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:38029
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:38029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.87.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:32:11.736862 12691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:32:11.737807 12691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:32:11.745904 12698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:11.745962 12697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:11.745932 12691 server_base.cc:1034] running on GCE node
W20260808 22:32:11.746099 12701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:11.746749 12691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:32:11.747700 12691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:32:11.748852 12691 hybrid_clock.cc:648] HybridClock initialized: now 1786228331748840 us; error 27 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-08T22:32:11Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1402 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:32:11 dist-test-slave-266d krb5kdc[12707](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:32:11 dist-test-slave-266d krb5kdc[12707](info): set up 2 sockets
Aug 08 22:32:11 dist-test-slave-266d krb5kdc[12707](info): commencing operation
krb5kdc: starting...
W20260808 22:32:12.896780 32094 mini_kdc.cc:121] Time spent starting KDC: real 1.110s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Aug 08 22:32:12 dist-test-slave-266d krb5kdc[12707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228332, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:32:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:32:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.87.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
I20260808 22:32:13.015468 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:45463
--webserver_interface=127.31.87.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:35945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:45463
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:32:13.122717 12727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:32:13.122960 12727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:32:13.122999 12727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:32:13.126879 12727 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:32:13.126925 12727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:32:13.126945 12727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:32:13.126976 12727 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:32:13.127002 12727 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:32:13.132012 12727 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:35945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:45463
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:45463
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.87.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12727
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:32:13.132937 12727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:32:13.133903 12727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:32:13.141800 12736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:13.141847 12737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:13.142088 12727 server_base.cc:1034] running on GCE node
W20260808 22:32:13.141860 12741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:13.142609 12727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:32:13.143620 12727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:32:13.144773 12727 hybrid_clock.cc:648] HybridClock initialized: now 1786228333144757 us; error 27 us; skew 500 ppm
Aug 08 22:32:13 dist-test-slave-266d krb5kdc[12707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228333, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:32:13.147331 12727 init.cc:377] Logged in from keytab as kudu/127.31.87.190@KRBTEST.COM (short username kudu)
I20260808 22:32:13.148083 12727 webserver.cc:460] Webserver started at http://127.31.87.190:40987/ using document root <none> and password file <none>
I20260808 22:32:13.148561 12727 fs_manager.cc:359] Metadata directory not provided
I20260808 22:32:13.148614 12727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:32:13.148773 12727 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:32:13.150689 12727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/master-0/data/instance:
uuid: "545fd04e60214fca9ac9e3cfceff3562"
format_stamp: "Formatted at 2026-08-08 22:32:13 on dist-test-slave-266d"
server_key: "a621170fda00b72aabb987badcb4540d"
server_key_iv: "dd455b0ccfc620538b36b30da49ea6c3"
server_key_version: "encryptionkey@0"
I20260808 22:32:13.151168 12727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/master-0/wal/instance:
uuid: "545fd04e60214fca9ac9e3cfceff3562"
format_stamp: "Formatted at 2026-08-08 22:32:13 on dist-test-slave-266d"
server_key: "a621170fda00b72aabb987badcb4540d"
server_key_iv: "dd455b0ccfc620538b36b30da49ea6c3"
server_key_version: "encryptionkey@0"
I20260808 22:32:13.154518 12727 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 22:32:13.156713 12752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:13.157869 12727 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 22:32:13.157966 12727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "545fd04e60214fca9ac9e3cfceff3562"
format_stamp: "Formatted at 2026-08-08 22:32:13 on dist-test-slave-266d"
server_key: "a621170fda00b72aabb987badcb4540d"
server_key_iv: "dd455b0ccfc620538b36b30da49ea6c3"
server_key_version: "encryptionkey@0"
I20260808 22:32:13.158043 12727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:32:13.167999 12727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:32:13.170512 12727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:32:13.170645 12727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:32:13.176831 12727 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.190:45463
I20260808 22:32:13.176873 12835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.190:45463 every 8 connection(s)
I20260808 22:32:13.177703 12727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:32:13.180275 12836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:32:13.180990 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 12727
I20260808 22:32:13.181157 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/master-0/wal/instance
I20260808 22:32:13.181488 32094 external_mini_cluster.cc:1428] Setting key 8c0b3d25f02a9d008193ad90f69e7e27
I20260808 22:32:13.186414 12836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 545fd04e60214fca9ac9e3cfceff3562: Bootstrap starting.
I20260808 22:32:13.188948 12836 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 545fd04e60214fca9ac9e3cfceff3562: Neither blocks nor log segments found. Creating new log.
I20260808 22:32:13.189785 12836 log.cc:824] T 00000000000000000000000000000000 P 545fd04e60214fca9ac9e3cfceff3562: Log is configured to *not* fsync() on all Append() calls
Aug 08 22:32:13 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228332, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:32:13.191465 12836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 545fd04e60214fca9ac9e3cfceff3562: No bootstrap required, opened a new log
I20260808 22:32:13.194577 12840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:13.182782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44692 (local address 127.31.87.190:45463)
0808 22:32:13.183142 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:13.183157 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:13.183190 (+ 33us) server_negotiation.cc:407] Connection header received
0808 22:32:13.183804 (+ 614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:13.183821 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:13.184113 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:13.184381 (+ 268us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:13.185283 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:13.186206 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:13.187341 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:13.187780 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:13.187889 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:13.190486 (+ 2597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:13.190492 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:13.190504 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:13.190530 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:13.192041 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:13.192432 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:13.192437 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:13.192441 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:13.192544 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:13.192797 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:13.192800 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:13.192802 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:13.193187 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:13.193354 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:13.193623 (+ 269us) server_negotiation.cc:299] Negotiation successful
0808 22:32:13.193893 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":103,"threads_started":1}
I20260808 22:32:13.194762 12836 raft_consensus.cc:348] T 00000000000000000000000000000000 P 545fd04e60214fca9ac9e3cfceff3562 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545fd04e60214fca9ac9e3cfceff3562" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45463 } }
I20260808 22:32:13.194983 12836 raft_consensus.cc:374] T 00000000000000000000000000000000 P 545fd04e60214fca9ac9e3cfceff3562 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:32:13.195034 12836 raft_consensus.cc:729] T 00000000000000000000000000000000 P 545fd04e60214fca9ac9e3cfceff3562 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 545fd04e60214fca9ac9e3cfceff3562, State: Initialized, Role: FOLLOWER
I20260808 22:32:13.195462 12836 consensus_queue.cc:260] T 00000000000000000000000000000000 P 545fd04e60214fca9ac9e3cfceff3562 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545fd04e60214fca9ac9e3cfceff3562" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45463 } }
I20260808 22:32:13.195593 12836 raft_consensus.cc:388] T 00000000000000000000000000000000 P 545fd04e60214fca9ac9e3cfceff3562 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:32:13.195662 12836 raft_consensus.cc:482] T 00000000000000000000000000000000 P 545fd04e60214fca9ac9e3cfceff3562 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:32:13.195757 12836 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 545fd04e60214fca9ac9e3cfceff3562 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:32:13.196597 12836 raft_consensus.cc:504] T 00000000000000000000000000000000 P 545fd04e60214fca9ac9e3cfceff3562 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545fd04e60214fca9ac9e3cfceff3562" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45463 } }
I20260808 22:32:13.196909 12836 leader_election.cc:302] T 00000000000000000000000000000000 P 545fd04e60214fca9ac9e3cfceff3562 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 545fd04e60214fca9ac9e3cfceff3562; no voters:
I20260808 22:32:13.197206 12836 leader_election.cc:288] T 00000000000000000000000000000000 P 545fd04e60214fca9ac9e3cfceff3562 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:32:13.197297 12845 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 545fd04e60214fca9ac9e3cfceff3562 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:32:13.197505 12845 raft_consensus.cc:686] T 00000000000000000000000000000000 P 545fd04e60214fca9ac9e3cfceff3562 [term 1 LEADER]: Becoming Leader. State: Replica: 545fd04e60214fca9ac9e3cfceff3562, State: Running, Role: LEADER
I20260808 22:32:13.197841 12845 consensus_queue.cc:237] T 00000000000000000000000000000000 P 545fd04e60214fca9ac9e3cfceff3562 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545fd04e60214fca9ac9e3cfceff3562" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45463 } }
I20260808 22:32:13.198398 12836 sys_catalog.cc:564] T 00000000000000000000000000000000 P 545fd04e60214fca9ac9e3cfceff3562 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:32:13.199491 12848 sys_catalog.cc:455] T 00000000000000000000000000000000 P 545fd04e60214fca9ac9e3cfceff3562 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 545fd04e60214fca9ac9e3cfceff3562. Latest consensus state: current_term: 1 leader_uuid: "545fd04e60214fca9ac9e3cfceff3562" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545fd04e60214fca9ac9e3cfceff3562" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45463 } } }
I20260808 22:32:13.199482 12846 sys_catalog.cc:455] T 00000000000000000000000000000000 P 545fd04e60214fca9ac9e3cfceff3562 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "545fd04e60214fca9ac9e3cfceff3562" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545fd04e60214fca9ac9e3cfceff3562" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45463 } } }
I20260808 22:32:13.199640 12848 sys_catalog.cc:458] T 00000000000000000000000000000000 P 545fd04e60214fca9ac9e3cfceff3562 [sys.catalog]: This master's current role is: LEADER
I20260808 22:32:13.199717 12846 sys_catalog.cc:458] T 00000000000000000000000000000000 P 545fd04e60214fca9ac9e3cfceff3562 [sys.catalog]: This master's current role is: LEADER
I20260808 22:32:13.200536 12864 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:32:13.203032 12864 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:32:13.207731 12864 catalog_manager.cc:1347] Generated new cluster ID: 16be0d934d28457f9875c10fd1a317b3
I20260808 22:32:13.207793 12864 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:32:13.220873 12864 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:32:13.222043 12864 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:32:13.227458 12864 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 545fd04e60214fca9ac9e3cfceff3562: Generated new TSK 0
I20260808 22:32:13.228194 12864 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.87.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab.
Entry for principal kudu/127.31.87.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab.
I20260808 22:32:13.262437 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.129:0
--local_ip_for_outbound_sockets=127.31.87.129
--webserver_interface=127.31.87.129
--webserver_port=0
--tserver_master_addrs=127.31.87.190:45463
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab
--principal=kudu/127.31.87.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:35945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:32:13.370872 12883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:32:13.371122 12883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:32:13.371182 12883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:32:13.375455 12883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:32:13.375511 12883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:32:13.375593 12883 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.129
I20260808 22:32:13.380702 12883 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:35945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab
--principal=kudu/127.31.87.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.87.129
--webserver_port=0
--tserver_master_addrs=127.31.87.190:45463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12883
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.129
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:32:13.381745 12883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:32:13.382715 12883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:32:13.391971 12896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:13.392020 12892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:13.392100 12883 server_base.cc:1034] running on GCE node
W20260808 22:32:13.391978 12891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:13.392629 12883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:32:13.393256 12883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:32:13.394407 12883 hybrid_clock.cc:648] HybridClock initialized: now 1786228333394392 us; error 26 us; skew 500 ppm
Aug 08 22:32:13 dist-test-slave-266d krb5kdc[12707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228333, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:32:13.396947 12883 init.cc:377] Logged in from keytab as kudu/127.31.87.129@KRBTEST.COM (short username kudu)
I20260808 22:32:13.397778 12883 webserver.cc:460] Webserver started at http://127.31.87.129:34897/ using document root <none> and password file <none>
I20260808 22:32:13.398264 12883 fs_manager.cc:359] Metadata directory not provided
I20260808 22:32:13.398317 12883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:32:13.398473 12883 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:32:13.400506 12883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-0/data/instance:
uuid: "080019271e784d2c8d6533797a7e05e1"
format_stamp: "Formatted at 2026-08-08 22:32:13 on dist-test-slave-266d"
server_key: "4ee1279b6174a907a6b89254956dfaaf"
server_key_iv: "3554899ea8bb13753092faae96f6a025"
server_key_version: "encryptionkey@0"
I20260808 22:32:13.401007 12883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-0/wal/instance:
uuid: "080019271e784d2c8d6533797a7e05e1"
format_stamp: "Formatted at 2026-08-08 22:32:13 on dist-test-slave-266d"
server_key: "4ee1279b6174a907a6b89254956dfaaf"
server_key_iv: "3554899ea8bb13753092faae96f6a025"
server_key_version: "encryptionkey@0"
I20260808 22:32:13.404433 12883 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 22:32:13.406904 12906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:13.408049 12883 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:32:13.408152 12883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-0/wal
uuid: "080019271e784d2c8d6533797a7e05e1"
format_stamp: "Formatted at 2026-08-08 22:32:13 on dist-test-slave-266d"
server_key: "4ee1279b6174a907a6b89254956dfaaf"
server_key_iv: "3554899ea8bb13753092faae96f6a025"
server_key_version: "encryptionkey@0"
I20260808 22:32:13.408232 12883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:32:13.429924 12883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:32:13.432521 12883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:32:13.432673 12883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:32:13.433218 12883 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:32:13.433995 12883 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:32:13.434041 12883 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:13.434077 12883 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:32:13.434098 12883 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:13.442521 12883 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.129:36205
I20260808 22:32:13.442538 13092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.129:36205 every 8 connection(s)
I20260808 22:32:13.443362 12883 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
I20260808 22:32:13.448513 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 12883
I20260808 22:32:13.448642 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-0/wal/instance
I20260808 22:32:13.448912 32094 external_mini_cluster.cc:1428] Setting key 64cb0db14b5e832d8c92b87ebf47d085
Aug 08 22:32:13 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228333, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.129@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:32:13.456300 12840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:13.445005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:45495 (local address 127.31.87.190:45463)
0808 22:32:13.445160 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:13.445164 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:13.445946 (+ 782us) server_negotiation.cc:407] Connection header received
0808 22:32:13.447178 (+ 1232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:13.447182 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:13.447241 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:13.447316 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:13.448417 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:13.448990 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:13.450187 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:13.450487 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:13.450628 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:13.452780 (+ 2152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:13.452784 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:13.452787 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:13.452814 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:13.454180 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:13.454742 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:13.454746 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:13.454747 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:13.454832 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:13.455212 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:13.455215 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:13.455217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:13.455407 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:13.455486 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:13.456073 (+ 587us) server_negotiation.cc:299] Negotiation successful
0808 22:32:13.456181 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 22:32:13.457373 13095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:13.445382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.129:45495)
0808 22:32:13.445806 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:13.445846 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:13.446976 (+ 1130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:13.447487 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:13.447493 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:13.447823 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:13.448261 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:13.448284 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:13.449125 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:13.449128 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:13.450055 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:13.450067 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:13.450625 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:13.450628 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:13.450775 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:13.451301 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:13.451316 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:13.452624 (+ 1308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:32:13.454292 (+ 1668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:13.454298 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:13.454311 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:13.454626 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:13.454940 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:13.454943 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:13.454945 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:13.455119 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:13.455505 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:32:13.455511 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:32:13.455809 (+ 298us) client_negotiation.cc:769] Sending connection context
0808 22:32:13.456016 (+ 207us) client_negotiation.cc:240] Negotiation successful
0808 22:32:13.456349 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":114,"threads_started":1}
I20260808 22:32:13.458530 13093 heartbeater.cc:347] Connected to a master server at 127.31.87.190:45463
I20260808 22:32:13.458791 13093 heartbeater.cc:464] Registering TS with master...
I20260808 22:32:13.459335 13093 heartbeater.cc:511] Master 127.31.87.190:45463 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.31.87.130@KRBTEST.COM; defaulting to no policy
I20260808 22:32:13.461012 12779 ts_manager.cc:194] Registered new tserver with Master: 080019271e784d2c8d6533797a7e05e1 (127.31.87.129:36205)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.130@KRBTEST.COM" created.
I20260808 22:32:13.462249 12779 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.129@KRBTEST.COM'} at 127.31.87.129:45495
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab.
Entry for principal kudu/127.31.87.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab.
I20260808 22:32:13.476075 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.130:0
--local_ip_for_outbound_sockets=127.31.87.130
--webserver_interface=127.31.87.130
--webserver_port=0
--tserver_master_addrs=127.31.87.190:45463
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab
--principal=kudu/127.31.87.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:35945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:32:13.581574 13098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:32:13.581845 13098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:32:13.581903 13098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:32:13.585798 13098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:32:13.585848 13098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:32:13.585923 13098 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.130
I20260808 22:32:13.590934 13098 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:35945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab
--principal=kudu/127.31.87.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.87.130
--webserver_port=0
--tserver_master_addrs=127.31.87.190:45463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13098
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.130
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:32:13.591837 13098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:32:13.592762 13098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:32:13.601702 13108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:13.601878 13098 server_base.cc:1034] running on GCE node
W20260808 22:32:13.601730 13106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:13.601702 13105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:13.602304 13098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:32:13.602856 13098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:32:13.604005 13098 hybrid_clock.cc:648] HybridClock initialized: now 1786228333603986 us; error 32 us; skew 500 ppm
Aug 08 22:32:13 dist-test-slave-266d krb5kdc[12707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228333, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:32:13.606534 13098 init.cc:377] Logged in from keytab as kudu/127.31.87.130@KRBTEST.COM (short username kudu)
I20260808 22:32:13.607335 13098 webserver.cc:460] Webserver started at http://127.31.87.130:32909/ using document root <none> and password file <none>
I20260808 22:32:13.607820 13098 fs_manager.cc:359] Metadata directory not provided
I20260808 22:32:13.607874 13098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:32:13.608033 13098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:32:13.609939 13098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-1/data/instance:
uuid: "bc4b68bec0c04de186fa0946d8584098"
format_stamp: "Formatted at 2026-08-08 22:32:13 on dist-test-slave-266d"
server_key: "36ef4d0dde2431c5ef84be438b3898c8"
server_key_iv: "b0caf63f98dd47f90ce547fe8590cc44"
server_key_version: "encryptionkey@0"
I20260808 22:32:13.610430 13098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-1/wal/instance:
uuid: "bc4b68bec0c04de186fa0946d8584098"
format_stamp: "Formatted at 2026-08-08 22:32:13 on dist-test-slave-266d"
server_key: "36ef4d0dde2431c5ef84be438b3898c8"
server_key_iv: "b0caf63f98dd47f90ce547fe8590cc44"
server_key_version: "encryptionkey@0"
I20260808 22:32:13.613857 13098 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 22:32:13.616061 13117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:13.617202 13098 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:32:13.617304 13098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-1/wal
uuid: "bc4b68bec0c04de186fa0946d8584098"
format_stamp: "Formatted at 2026-08-08 22:32:13 on dist-test-slave-266d"
server_key: "36ef4d0dde2431c5ef84be438b3898c8"
server_key_iv: "b0caf63f98dd47f90ce547fe8590cc44"
server_key_version: "encryptionkey@0"
I20260808 22:32:13.617393 13098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:32:13.644555 13098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:32:13.647532 13098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:32:13.647701 13098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:32:13.648257 13098 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:32:13.649040 13098 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:32:13.649101 13098 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:13.649159 13098 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:32:13.649189 13098 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:13.657437 13098 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.130:33389
I20260808 22:32:13.657466 13294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.130:33389 every 8 connection(s)
I20260808 22:32:13.658244 13098 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
I20260808 22:32:13.661976 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 13098
I20260808 22:32:13.662114 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-1/wal/instance
I20260808 22:32:13.662460 32094 external_mini_cluster.cc:1428] Setting key 1cc56727f40e1befc5ae9469a112b2e2
Aug 08 22:32:13 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228333, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.130@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:32:13.670971 12840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:13.659954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:58501 (local address 127.31.87.190:45463)
0808 22:32:13.660082 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:13.660086 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:13.660881 (+ 795us) server_negotiation.cc:407] Connection header received
0808 22:32:13.662177 (+ 1296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:13.662181 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:13.662238 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:13.662324 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:13.663510 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:13.664119 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:13.665226 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:13.665456 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:13.665543 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:13.667726 (+ 2183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:13.667730 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:13.667733 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:13.667759 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:13.669060 (+ 1301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:13.669554 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:13.669558 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:13.669560 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:13.669637 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:13.669938 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:13.669942 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:13.669944 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:13.670121 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:13.670205 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:13.670678 (+ 473us) server_negotiation.cc:299] Negotiation successful
0808 22:32:13.670811 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
WARNING: no policy specified for kudu/127.31.87.131@KRBTEST.COM; defaulting to no policy
I20260808 22:32:13.671722 13303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:13.660309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.130:58501)
0808 22:32:13.660732 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:13.660777 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:13.661966 (+ 1189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:13.662482 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:13.662489 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:13.662859 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:13.663373 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:13.663388 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:13.664237 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:13.664240 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:13.665122 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:13.665134 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:13.665587 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:13.665591 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:13.665750 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:13.666247 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:13.666269 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:13.667577 (+ 1308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:32:13.669172 (+ 1595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:13.669178 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:13.669192 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:13.669462 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:13.669717 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:13.669720 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:13.669722 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:13.669853 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:13.670215 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:32:13.670221 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:32:13.670458 (+ 237us) client_negotiation.cc:769] Sending connection context
0808 22:32:13.670621 (+ 163us) client_negotiation.cc:240] Negotiation successful
0808 22:32:13.670926 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":109,"threads_started":1}
I20260808 22:32:13.672821 13296 heartbeater.cc:347] Connected to a master server at 127.31.87.190:45463
I20260808 22:32:13.673040 13296 heartbeater.cc:464] Registering TS with master...
I20260808 22:32:13.673514 13296 heartbeater.cc:511] Master 127.31.87.190:45463 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.131@KRBTEST.COM" created.
I20260808 22:32:13.674505 12779 ts_manager.cc:194] Registered new tserver with Master: bc4b68bec0c04de186fa0946d8584098 (127.31.87.130:33389)
I20260808 22:32:13.675062 12779 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.130@KRBTEST.COM'} at 127.31.87.130:58501
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab.
Entry for principal kudu/127.31.87.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab.
I20260808 22:32:13.687952 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.131:0
--local_ip_for_outbound_sockets=127.31.87.131
--webserver_interface=127.31.87.131
--webserver_port=0
--tserver_master_addrs=127.31.87.190:45463
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab
--principal=kudu/127.31.87.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:35945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:32:13.800066 13306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:32:13.800299 13306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:32:13.800343 13306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:32:13.804260 13306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:32:13.804313 13306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:32:13.804380 13306 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.131
I20260808 22:32:13.809469 13306 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:35945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab
--principal=kudu/127.31.87.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.87.131
--webserver_port=0
--tserver_master_addrs=127.31.87.190:45463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13306
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.131
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:32:13.810355 13306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:32:13.811303 13306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:32:13.820526 13323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:13.820541 13320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:13.820662 13319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:13.820981 13306 server_base.cc:1034] running on GCE node
I20260808 22:32:13.821424 13306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:32:13.822049 13306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:32:13.823230 13306 hybrid_clock.cc:648] HybridClock initialized: now 1786228333823209 us; error 25 us; skew 500 ppm
Aug 08 22:32:13 dist-test-slave-266d krb5kdc[12707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228333, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:32:13.825907 13306 init.cc:377] Logged in from keytab as kudu/127.31.87.131@KRBTEST.COM (short username kudu)
I20260808 22:32:13.826754 13306 webserver.cc:460] Webserver started at http://127.31.87.131:37889/ using document root <none> and password file <none>
I20260808 22:32:13.827297 13306 fs_manager.cc:359] Metadata directory not provided
I20260808 22:32:13.827364 13306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:32:13.827565 13306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:32:13.829664 13306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-2/data/instance:
uuid: "55c1ec59ec624cd2b58b719f7a002b36"
format_stamp: "Formatted at 2026-08-08 22:32:13 on dist-test-slave-266d"
server_key: "e1308fc6e46ade3ef7ebef2198662d99"
server_key_iv: "38c69d63fb0d46bcf53ec240a40e4fa2"
server_key_version: "encryptionkey@0"
I20260808 22:32:13.830243 13306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-2/wal/instance:
uuid: "55c1ec59ec624cd2b58b719f7a002b36"
format_stamp: "Formatted at 2026-08-08 22:32:13 on dist-test-slave-266d"
server_key: "e1308fc6e46ade3ef7ebef2198662d99"
server_key_iv: "38c69d63fb0d46bcf53ec240a40e4fa2"
server_key_version: "encryptionkey@0"
I20260808 22:32:13.833949 13306 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 22:32:13.836305 13333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:13.837504 13306 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 22:32:13.837615 13306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-2/wal
uuid: "55c1ec59ec624cd2b58b719f7a002b36"
format_stamp: "Formatted at 2026-08-08 22:32:13 on dist-test-slave-266d"
server_key: "e1308fc6e46ade3ef7ebef2198662d99"
server_key_iv: "38c69d63fb0d46bcf53ec240a40e4fa2"
server_key_version: "encryptionkey@0"
I20260808 22:32:13.837709 13306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:32:13.877058 13306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:32:13.879863 13306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:32:13.880049 13306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:32:13.880654 13306 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:32:13.881526 13306 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:32:13.881577 13306 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:13.881629 13306 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:32:13.881659 13306 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:13.890275 13306 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.131:35765
I20260808 22:32:13.890305 13518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.131:35765 every 8 connection(s)
I20260808 22:32:13.891139 13306 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
I20260808 22:32:13.894268 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 13306
I20260808 22:32:13.894399 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-2/wal/instance
I20260808 22:32:13.894675 32094 external_mini_cluster.cc:1428] Setting key cb1aa5ecce40f414ddc1c50bb24c07b3
Aug 08 22:32:13 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228333, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.131@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:32:13.903724 12840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:13.892829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:56197 (local address 127.31.87.190:45463)
0808 22:32:13.892951 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:13.892955 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:13.893794 (+ 839us) server_negotiation.cc:407] Connection header received
0808 22:32:13.895039 (+ 1245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:13.895043 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:13.895093 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:13.895192 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:13.896426 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:13.897030 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:13.898132 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:13.898359 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:13.898445 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:13.900757 (+ 2312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:13.900760 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:13.900763 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:13.900788 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:13.901922 (+ 1134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:13.902361 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:13.902365 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:13.902366 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:13.902439 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:13.902733 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:13.902737 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:13.902738 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:13.902908 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:13.902993 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:13.903450 (+ 457us) server_negotiation.cc:299] Negotiation successful
0808 22:32:13.903566 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 22:32:13.904451 13522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:13.893206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.131:56197)
0808 22:32:13.893635 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:13.893677 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:13.894832 (+ 1155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:13.895350 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:13.895357 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:13.895743 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:13.896270 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:13.896284 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:13.897158 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:13.897161 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:13.898018 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:13.898028 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:13.898490 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:13.898494 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:13.898650 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:13.899181 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:13.899197 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:13.900612 (+ 1415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:32:13.902022 (+ 1410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:13.902028 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:13.902042 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:13.902273 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:13.902519 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:13.902522 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:13.902524 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:13.902650 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:13.903002 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:32:13.903009 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:32:13.903232 (+ 223us) client_negotiation.cc:769] Sending connection context
0808 22:32:13.903390 (+ 158us) client_negotiation.cc:240] Negotiation successful
0808 22:32:13.903683 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":109,"threads_started":1}
I20260808 22:32:13.905431 13519 heartbeater.cc:347] Connected to a master server at 127.31.87.190:45463
I20260808 22:32:13.905630 13519 heartbeater.cc:464] Registering TS with master...
I20260808 22:32:13.906051 13519 heartbeater.cc:511] Master 127.31.87.190:45463 requested a full tablet report, sending...
I20260808 22:32:13.906996 12779 ts_manager.cc:194] Registered new tserver with Master: 55c1ec59ec624cd2b58b719f7a002b36 (127.31.87.131:35765)
I20260808 22:32:13.907559 12779 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.131@KRBTEST.COM'} at 127.31.87.131:56197
I20260808 22:32:13.908175 32094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.31.87.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.87.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.87.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.129.keytab.
Entry for principal oryx/127.31.87.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.129.keytab.
WARNING: no policy specified for oryx/127.31.87.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.87.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.87.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.130.keytab.
Entry for principal oryx/127.31.87.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.130.keytab.
WARNING: no policy specified for oryx/127.31.87.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.87.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.87.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.131.keytab.
Entry for principal oryx/127.31.87.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.131.keytab.
I20260808 22:32:13.973656 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 12883
I20260808 22:32:13.985316 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 13098
I20260808 22:32:14.014451 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 13306
I20260808 22:32:14.024225 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 12727
I20260808 22:32:14.035930 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:45463
--webserver_interface=127.31.87.190
--webserver_port=40987
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:35945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:45463
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:32:14.142148 13532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:32:14.142374 13532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:32:14.142413 13532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:32:14.146469 13532 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:32:14.146521 13532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:32:14.146539 13532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:32:14.146554 13532 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:32:14.146569 13532 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:32:14.151597 13532 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:35945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:45463
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:45463
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.87.190
--webserver_port=40987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13532
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:32:14.152499 13532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:32:14.153448 13532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 22:32:14.161491 13532 server_base.cc:1034] running on GCE node
W20260808 22:32:14.161435 13546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:14.161422 13539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:14.161448 13538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:14.162276 13532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:32:14.163282 13532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:32:14.164446 13532 hybrid_clock.cc:648] HybridClock initialized: now 1786228334164428 us; error 30 us; skew 500 ppm
Aug 08 22:32:14 dist-test-slave-266d krb5kdc[12707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228334, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:32:14.167050 13532 init.cc:377] Logged in from keytab as kudu/127.31.87.190@KRBTEST.COM (short username kudu)
I20260808 22:32:14.167847 13532 webserver.cc:460] Webserver started at http://127.31.87.190:40987/ using document root <none> and password file <none>
I20260808 22:32:14.168352 13532 fs_manager.cc:359] Metadata directory not provided
I20260808 22:32:14.168428 13532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:32:14.172024 13532 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:32:14.174002 13557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:14.175136 13532 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:32:14.175244 13532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "545fd04e60214fca9ac9e3cfceff3562"
format_stamp: "Formatted at 2026-08-08 22:32:13 on dist-test-slave-266d"
server_key: "a621170fda00b72aabb987badcb4540d"
server_key_iv: "dd455b0ccfc620538b36b30da49ea6c3"
server_key_version: "encryptionkey@0"
I20260808 22:32:14.175732 13532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:32:14.186012 13532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:32:14.188448 13532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:32:14.188611 13532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:32:14.194999 13532 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.190:45463
I20260808 22:32:14.195011 13649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.190:45463 every 8 connection(s)
I20260808 22:32:14.195956 13532 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:32:14.199982 13650 sys_catalog.cc:263] Verifying existing consensus state
I20260808 22:32:14.201463 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 13532
I20260808 22:32:14.201756 13650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 545fd04e60214fca9ac9e3cfceff3562: Bootstrap starting.
I20260808 22:32:14.201985 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.129:36205
--local_ip_for_outbound_sockets=127.31.87.129
--tserver_master_addrs=127.31.87.190:45463
--webserver_port=34897
--webserver_interface=127.31.87.129
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab
--principal=kudu/127.31.87.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:35945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.31.87.129
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 22:32:14.218235 13650 log.cc:824] T 00000000000000000000000000000000 P 545fd04e60214fca9ac9e3cfceff3562: Log is configured to *not* fsync() on all Append() calls
I20260808 22:32:14.224673 13650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 545fd04e60214fca9ac9e3cfceff3562: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 22:32:14.225004 13650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 545fd04e60214fca9ac9e3cfceff3562: Bootstrap complete.
I20260808 22:32:14.228989 13650 raft_consensus.cc:348] T 00000000000000000000000000000000 P 545fd04e60214fca9ac9e3cfceff3562 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545fd04e60214fca9ac9e3cfceff3562" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45463 } }
I20260808 22:32:14.229419 13650 raft_consensus.cc:729] T 00000000000000000000000000000000 P 545fd04e60214fca9ac9e3cfceff3562 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 545fd04e60214fca9ac9e3cfceff3562, State: Initialized, Role: FOLLOWER
I20260808 22:32:14.229876 13650 consensus_queue.cc:260] T 00000000000000000000000000000000 P 545fd04e60214fca9ac9e3cfceff3562 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545fd04e60214fca9ac9e3cfceff3562" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45463 } }
I20260808 22:32:14.229985 13650 raft_consensus.cc:388] T 00000000000000000000000000000000 P 545fd04e60214fca9ac9e3cfceff3562 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:32:14.230042 13650 raft_consensus.cc:482] T 00000000000000000000000000000000 P 545fd04e60214fca9ac9e3cfceff3562 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:32:14.230139 13650 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 545fd04e60214fca9ac9e3cfceff3562 [term 1 FOLLOWER]: Advancing to term 2
I20260808 22:32:14.231917 13650 raft_consensus.cc:504] T 00000000000000000000000000000000 P 545fd04e60214fca9ac9e3cfceff3562 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545fd04e60214fca9ac9e3cfceff3562" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45463 } }
I20260808 22:32:14.232223 13650 leader_election.cc:302] T 00000000000000000000000000000000 P 545fd04e60214fca9ac9e3cfceff3562 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 545fd04e60214fca9ac9e3cfceff3562; no voters:
I20260808 22:32:14.232497 13650 leader_election.cc:288] T 00000000000000000000000000000000 P 545fd04e60214fca9ac9e3cfceff3562 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 22:32:14.232594 13656 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 545fd04e60214fca9ac9e3cfceff3562 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 22:32:14.232802 13656 raft_consensus.cc:686] T 00000000000000000000000000000000 P 545fd04e60214fca9ac9e3cfceff3562 [term 2 LEADER]: Becoming Leader. State: Replica: 545fd04e60214fca9ac9e3cfceff3562, State: Running, Role: LEADER
I20260808 22:32:14.233186 13656 consensus_queue.cc:237] T 00000000000000000000000000000000 P 545fd04e60214fca9ac9e3cfceff3562 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545fd04e60214fca9ac9e3cfceff3562" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45463 } }
I20260808 22:32:14.233722 13650 sys_catalog.cc:564] T 00000000000000000000000000000000 P 545fd04e60214fca9ac9e3cfceff3562 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:32:14.234104 13658 sys_catalog.cc:455] T 00000000000000000000000000000000 P 545fd04e60214fca9ac9e3cfceff3562 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 545fd04e60214fca9ac9e3cfceff3562. Latest consensus state: current_term: 2 leader_uuid: "545fd04e60214fca9ac9e3cfceff3562" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545fd04e60214fca9ac9e3cfceff3562" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45463 } } }
I20260808 22:32:14.234228 13658 sys_catalog.cc:458] T 00000000000000000000000000000000 P 545fd04e60214fca9ac9e3cfceff3562 [sys.catalog]: This master's current role is: LEADER
I20260808 22:32:14.234570 13657 sys_catalog.cc:455] T 00000000000000000000000000000000 P 545fd04e60214fca9ac9e3cfceff3562 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "545fd04e60214fca9ac9e3cfceff3562" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545fd04e60214fca9ac9e3cfceff3562" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45463 } } }
I20260808 22:32:14.234684 13657 sys_catalog.cc:458] T 00000000000000000000000000000000 P 545fd04e60214fca9ac9e3cfceff3562 [sys.catalog]: This master's current role is: LEADER
I20260808 22:32:14.235160 13662 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:32:14.238303 13662 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:32:14.238967 13662 catalog_manager.cc:1259] Loaded cluster ID: 16be0d934d28457f9875c10fd1a317b3
I20260808 22:32:14.239030 13662 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:32:14.241343 13662 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:32:14.241999 13662 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 545fd04e60214fca9ac9e3cfceff3562: Loaded TSK: 0
I20260808 22:32:14.243176 13662 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 22:32:14.314420 13653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:32:14.314666 13653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:32:14.314724 13653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:32:14.318658 13653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:32:14.318713 13653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:32:14.318789 13653 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.129
I20260808 22:32:14.323791 13653 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:35945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.129.keytab
--principal=oryx/127.31.87.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.129:36205
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.87.129
--webserver_port=34897
--tserver_master_addrs=127.31.87.190:45463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13653
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.129
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:32:14.324749 13653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:32:14.325723 13653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:32:14.336027 13690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:14.336027 13692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:14.336272 13653 server_base.cc:1034] running on GCE node
W20260808 22:32:14.336453 13694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:14.336846 13653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:32:14.337502 13653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:32:14.338677 13653 hybrid_clock.cc:648] HybridClock initialized: now 1786228334338657 us; error 35 us; skew 500 ppm
Aug 08 22:32:14 dist-test-slave-266d krb5kdc[12707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228334, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.87.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:32:14.341452 13653 init.cc:377] Logged in from keytab as oryx/127.31.87.129@KRBTEST.COM (short username oryx)
I20260808 22:32:14.342348 13653 webserver.cc:460] Webserver started at http://127.31.87.129:34897/ using document root <none> and password file <none>
I20260808 22:32:14.342875 13653 fs_manager.cc:359] Metadata directory not provided
I20260808 22:32:14.342958 13653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:32:14.346909 13653 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 22:32:14.348888 13704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:14.350128 13653 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:32:14.350242 13653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-0/wal
uuid: "080019271e784d2c8d6533797a7e05e1"
format_stamp: "Formatted at 2026-08-08 22:32:13 on dist-test-slave-266d"
server_key: "4ee1279b6174a907a6b89254956dfaaf"
server_key_iv: "3554899ea8bb13753092faae96f6a025"
server_key_version: "encryptionkey@0"
I20260808 22:32:14.350764 13653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:32:14.377959 13653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:32:14.380923 13653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:32:14.381127 13653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:32:14.381742 13653 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:32:14.382555 13653 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:32:14.382601 13653 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:14.382644 13653 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:32:14.382675 13653 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:14.391091 13653 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.129:36205
I20260808 22:32:14.391124 13890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.129:36205 every 8 connection(s)
I20260808 22:32:14.392031 13653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
I20260808 22:32:14.399219 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 13653
I20260808 22:32:14.399621 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.130:33389
--local_ip_for_outbound_sockets=127.31.87.130
--tserver_master_addrs=127.31.87.190:45463
--webserver_port=32909
--webserver_interface=127.31.87.130
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab
--principal=kudu/127.31.87.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:35945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.31.87.130
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 22:32:14 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.129@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:32:14 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.129@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 22:32:14.404575 13894 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:14.394015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:43919 (local address 127.31.87.190:45463)
0808 22:32:14.394519 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:14.394539 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:14.394730 (+ 191us) server_negotiation.cc:407] Connection header received
0808 22:32:14.396142 (+ 1412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:14.396153 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:14.396445 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:14.396739 (+ 294us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:14.397854 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:14.398799 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:14.399987 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:14.400438 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:14.400584 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:14.403912 (+ 3328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.87.129:43919: BlockingRecv error: recv got EOF from 127.31.87.129:43919 (error 108)
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":127,"threads_started":1}
I20260808 22:32:14.404644 13896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:14.394114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.129:43919)
0808 22:32:14.394573 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:14.394624 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:14.395746 (+ 1122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:14.396835 (+ 1089us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:14.396843 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:14.397258 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:14.397690 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:14.397703 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:14.398928 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:14.398931 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:14.399871 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:14.399882 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:14.400589 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:14.400593 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:14.400776 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:14.401307 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:14.401324 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:14.403727 (+ 2403us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":130,"threads_started":1}
W20260808 22:32:14.405045 13891 heartbeater.cc:644] Failed to heartbeat to 127.31.87.190:45463 (0 consecutive failures): Not authorized: Failed to ping master at 127.31.87.190:45463: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
Aug 08 22:32:14 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.129@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:32:14 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.129@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:32:14.410255 13896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:14.405469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.129:38705)
0808 22:32:14.405565 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:14.405578 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:14.405654 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:14.405867 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:14.405870 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:14.406023 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:14.406132 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:14.406140 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:14.406789 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:14.406792 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:14.407614 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:14.407626 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:14.407991 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:14.407995 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:14.408090 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:14.408476 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:14.408491 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:14.410144 (+ 1653us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":26}
W20260808 22:32:14.410502 13894 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:14.405549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:38705 (local address 127.31.87.190:45463)
0808 22:32:14.405648 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:14.405651 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:14.405663 (+ 12us) server_negotiation.cc:407] Connection header received
0808 22:32:14.405735 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:14.405737 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:14.405772 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:14.405831 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:14.406234 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:14.406694 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:14.407715 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:14.407903 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:14.407966 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:14.410261 (+ 2295us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.87.129:38705: BlockingRecv error: recv got EOF from 127.31.87.129:38705 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 08 22:32:14 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.129@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:32:14 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.129@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:32:14.415449 13896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:14.410675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.129:53753)
0808 22:32:14.410772 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:14.410785 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:14.410951 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:14.411179 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:14.411181 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:14.411294 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:14.411356 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:14.411361 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:14.412282 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:14.412284 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:14.412769 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:14.412774 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:14.413293 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:14.413295 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:14.413348 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:14.413764 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:14.413774 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:14.415372 (+ 1598us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 22:32:14.415606 13894 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:14.410874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:53753 (local address 127.31.87.190:45463)
0808 22:32:14.410987 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:14.410992 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:14.411006 (+ 14us) server_negotiation.cc:407] Connection header received
0808 22:32:14.411040 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:14.411043 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:14.411084 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:14.411177 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:14.411449 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:14.412199 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:14.412852 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:14.413208 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:14.413299 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:14.415507 (+ 2208us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.87.129:53753: BlockingRecv error: recv got EOF from 127.31.87.129:53753 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260808 22:32:14.415637 13891 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 22:32:14.514293 13897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:32:14.514539 13897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:32:14.514606 13897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:32:14.518538 13897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:32:14.518596 13897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:32:14.518671 13897 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.130
I20260808 22:32:14.523676 13897 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:35945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.130.keytab
--principal=oryx/127.31.87.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.130:33389
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.87.130
--webserver_port=32909
--tserver_master_addrs=127.31.87.190:45463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.130
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:32:14.524644 13897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:32:14.525614 13897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:32:14.534837 13904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:14.534880 13907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:14.534909 13905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:14.535354 13897 server_base.cc:1034] running on GCE node
I20260808 22:32:14.535874 13897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:32:14.536608 13897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:32:14.537788 13897 hybrid_clock.cc:648] HybridClock initialized: now 1786228334537763 us; error 33 us; skew 500 ppm
Aug 08 22:32:14 dist-test-slave-266d krb5kdc[12707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228334, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.87.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:32:14.541057 13897 init.cc:377] Logged in from keytab as oryx/127.31.87.130@KRBTEST.COM (short username oryx)
I20260808 22:32:14.542163 13897 webserver.cc:460] Webserver started at http://127.31.87.130:32909/ using document root <none> and password file <none>
I20260808 22:32:14.542830 13897 fs_manager.cc:359] Metadata directory not provided
I20260808 22:32:14.542924 13897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:32:14.547889 13897 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 22:32:14.550360 13920 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:14.551753 13897 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 22:32:14.551867 13897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-1/wal
uuid: "bc4b68bec0c04de186fa0946d8584098"
format_stamp: "Formatted at 2026-08-08 22:32:13 on dist-test-slave-266d"
server_key: "36ef4d0dde2431c5ef84be438b3898c8"
server_key_iv: "b0caf63f98dd47f90ce547fe8590cc44"
server_key_version: "encryptionkey@0"
I20260808 22:32:14.552553 13897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:32:14.568461 13897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:32:14.571782 13897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:32:14.571964 13897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:32:14.572638 13897 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:32:14.573647 13897 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:32:14.573705 13897 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:14.573766 13897 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:32:14.573796 13897 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:14.584705 13897 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.130:33389
I20260808 22:32:14.584712 14082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.130:33389 every 8 connection(s)
I20260808 22:32:14.585952 13897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
I20260808 22:32:14.586586 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 13897
I20260808 22:32:14.587067 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.131:35765
--local_ip_for_outbound_sockets=127.31.87.131
--tserver_master_addrs=127.31.87.190:45463
--webserver_port=37889
--webserver_interface=127.31.87.131
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab
--principal=kudu/127.31.87.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:35945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.31.87.131
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 22:32:14 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.130@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:32:14 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.130@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 22:32:14.597390 13894 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:14.589602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:50217 (local address 127.31.87.190:45463)
0808 22:32:14.589753 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:14.589758 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:14.589776 (+ 18us) server_negotiation.cc:407] Connection header received
0808 22:32:14.591047 (+ 1271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:14.591051 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:14.591112 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:14.591187 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:14.592317 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:14.592829 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:14.593942 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:14.594154 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:14.594245 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:14.597251 (+ 3006us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.87.130:50217: BlockingRecv error: recv got EOF from 127.31.87.130:50217 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 22:32:14.598045 14089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:14.588435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.130:50217)
0808 22:32:14.589264 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:14.589318 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:14.590849 (+ 1531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:14.591375 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:14.591381 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:14.591712 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:14.592168 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:14.592182 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:14.592946 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:14.592949 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:14.593827 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:14.593837 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:14.594289 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:14.594294 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:14.594453 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:14.594952 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:14.594968 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:14.597079 (+ 2111us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":148,"threads_started":1}
W20260808 22:32:14.598460 14084 heartbeater.cc:644] Failed to heartbeat to 127.31.87.190:45463 (0 consecutive failures): Not authorized: Failed to ping master at 127.31.87.190:45463: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
Aug 08 22:32:14 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.130@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:32:14 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.130@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 22:32:14.612846 13894 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:14.598933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:33613 (local address 127.31.87.190:45463)
0808 22:32:14.599026 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:14.599030 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:14.601256 (+ 2226us) server_negotiation.cc:407] Connection header received
0808 22:32:14.601381 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:14.601384 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:14.601421 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:14.601480 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:14.601828 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:14.602296 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:14.603056 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:14.603255 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:14.603337 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:14.612729 (+ 9392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.87.130:33613: BlockingRecv error: recv got EOF from 127.31.87.130:33613 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 22:32:14.613358 14089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:14.598861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.130:33613)
0808 22:32:14.601189 (+ 2328us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:14.601200 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:14.601289 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:14.601503 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:14.601506 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:14.601646 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:14.601728 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:14.601733 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:14.602393 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:14.602396 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:14.602941 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:14.602951 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:14.603345 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:14.603348 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:14.603431 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:14.603827 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:14.603839 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:14.613230 (+ 9391us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":2269}
Aug 08 22:32:14 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.130@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:32:14 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.130@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 22:32:14.620209 13894 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:14.614349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:55245 (local address 127.31.87.190:45463)
0808 22:32:14.614461 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:14.614465 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:14.614604 (+ 139us) server_negotiation.cc:407] Connection header received
0808 22:32:14.614764 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:14.614767 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:14.614816 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:14.614873 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:14.615394 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:14.616129 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:14.617258 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:14.617589 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:14.617658 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:14.620099 (+ 2441us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.87.130:55245: BlockingRecv error: recv got EOF from 127.31.87.130:55245 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 22:32:14.620635 14089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:14.614110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.130:55245)
0808 22:32:14.614542 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:14.614555 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:14.614662 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:14.615009 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:14.615012 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:14.615165 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:14.615249 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:14.615256 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:14.616280 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:14.616283 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:14.617125 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:14.617136 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:14.617798 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:14.617801 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:14.617879 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:14.618298 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:14.618312 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:14.620524 (+ 2212us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":369}
W20260808 22:32:14.620929 14084 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 22:32:14.743476 14088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:32:14.743695 14088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:32:14.743732 14088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:32:14.747918 14088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:32:14.747973 14088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:32:14.748031 14088 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.131
I20260808 22:32:14.753057 14088 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:35945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/krb5kdc/oryx_127.31.87.131.keytab
--principal=oryx/127.31.87.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.131:35765
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.87.131
--webserver_port=37889
--tserver_master_addrs=127.31.87.190:45463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14088
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.131
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:32:14.754169 14088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:32:14.755169 14088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:32:14.764582 14097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:14.764607 14098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:14.764621 14104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:14.765056 14088 server_base.cc:1034] running on GCE node
I20260808 22:32:14.765518 14088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:32:14.766255 14088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:32:14.767453 14088 hybrid_clock.cc:648] HybridClock initialized: now 1786228334767423 us; error 35 us; skew 500 ppm
Aug 08 22:32:14 dist-test-slave-266d krb5kdc[12707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228334, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.87.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:32:14.770686 14088 init.cc:377] Logged in from keytab as oryx/127.31.87.131@KRBTEST.COM (short username oryx)
I20260808 22:32:14.771529 14088 webserver.cc:460] Webserver started at http://127.31.87.131:37889/ using document root <none> and password file <none>
I20260808 22:32:14.772059 14088 fs_manager.cc:359] Metadata directory not provided
I20260808 22:32:14.772132 14088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:32:14.776017 14088 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 22:32:14.778210 14114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:14.779485 14088 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:32:14.779608 14088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-2/wal
uuid: "55c1ec59ec624cd2b58b719f7a002b36"
format_stamp: "Formatted at 2026-08-08 22:32:13 on dist-test-slave-266d"
server_key: "e1308fc6e46ade3ef7ebef2198662d99"
server_key_iv: "38c69d63fb0d46bcf53ec240a40e4fa2"
server_key_version: "encryptionkey@0"
I20260808 22:32:14.780162 14088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:32:14.801968 14088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:32:14.804942 14088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:32:14.805135 14088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:32:14.805704 14088 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:32:14.806522 14088 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:32:14.806569 14088 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:14.806622 14088 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:32:14.806650 14088 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:14.814934 14088 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.131:35765
I20260808 22:32:14.814988 14292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.131:35765 every 8 connection(s)
I20260808 22:32:14.815896 14088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
I20260808 22:32:14.821295 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 14088
Aug 08 22:32:14 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.131@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:32:14 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.131@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 22:32:14.826985 13894 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:14.817613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:33567 (local address 127.31.87.190:45463)
0808 22:32:14.817737 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:14.817741 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:14.818527 (+ 786us) server_negotiation.cc:407] Connection header received
0808 22:32:14.819725 (+ 1198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:14.819728 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:14.819785 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:14.819855 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:14.821018 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:14.821755 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:14.823209 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:14.823423 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:14.823518 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:14.826860 (+ 3342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.87.131:33567: BlockingRecv error: recv got EOF from 127.31.87.131:33567 (error 108)
Metrics: {"mutex_wait_us":70,"server-negotiator.queue_time_us":46}
I20260808 22:32:14.827646 14295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:14.817934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.131:33567)
0808 22:32:14.818369 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:14.818412 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:14.819523 (+ 1111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:14.820005 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:14.820011 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:14.820353 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:14.820877 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:14.820892 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:14.821904 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:14.821906 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:14.823088 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:14.823101 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:14.823570 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:14.823574 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:14.823710 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:14.824277 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:14.824297 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:14.826741 (+ 2444us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":111,"threads_started":1}
W20260808 22:32:14.828330 14293 heartbeater.cc:644] Failed to heartbeat to 127.31.87.190:45463 (0 consecutive failures): Not authorized: Failed to ping master at 127.31.87.190:45463: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
I20260808 22:32:14.830960 14297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:14.822334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44696 (local address 127.31.87.190:45463)
0808 22:32:14.822528 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:14.822531 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:14.822543 (+ 12us) server_negotiation.cc:407] Connection header received
0808 22:32:14.822576 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:14.822578 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:14.822616 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:14.822691 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:14.823515 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:14.824201 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:14.825350 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:14.825636 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:14.825729 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:14.826973 (+ 1244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:14.826979 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:14.826992 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:14.827020 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:14.828939 (+ 1919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:14.829445 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:14.829452 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:14.829458 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:14.829578 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:14.829829 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:14.829832 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:14.829834 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:14.830161 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:14.830344 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:14.830617 (+ 273us) server_negotiation.cc:299] Negotiation successful
0808 22:32:14.830845 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":135,"thread_start_us":55,"threads_started":1}
Aug 08 22:32:14 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.131@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:32:14 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.131@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:32:14.833696 14295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:14.828744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.131:37005)
0808 22:32:14.829228 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:14.829239 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:14.829315 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:14.829508 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:14.829511 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:14.829654 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:14.829736 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:14.829741 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:14.830392 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:14.830395 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:14.831001 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:14.831012 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:14.831524 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:14.831527 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:14.831592 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:14.831963 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:14.831976 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:14.833575 (+ 1599us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":83,"threads_started":1}
W20260808 22:32:14.833837 13894 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:14.828855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:37005 (local address 127.31.87.190:45463)
0808 22:32:14.829012 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:14.829016 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:14.829299 (+ 283us) server_negotiation.cc:407] Connection header received
0808 22:32:14.829383 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:14.829386 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:14.829424 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:14.829482 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:14.829835 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:14.830297 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:14.831142 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:14.831430 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:14.831526 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:14.833724 (+ 2198us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.87.131:37005: BlockingRecv error: recv got EOF from 127.31.87.131:37005 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 08 22:32:14 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.131@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:32:14 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.131@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:32:14.838507 14295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:14.834221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.131:55601)
0808 22:32:14.834306 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:14.834317 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:14.834375 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:14.834553 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:14.834556 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:14.834680 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:14.834762 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:14.834767 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:14.835407 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:14.835410 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:14.835932 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:14.835941 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:14.836403 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:14.836406 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:14.836488 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:14.836909 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:14.836924 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:14.838398 (+ 1474us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":26}
W20260808 22:32:14.838594 13894 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:14.834284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:55601 (local address 127.31.87.190:45463)
0808 22:32:14.834396 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:14.834400 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:14.834413 (+ 13us) server_negotiation.cc:407] Connection header received
0808 22:32:14.834438 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:14.834440 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:14.834475 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:14.834542 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:14.834845 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:14.835320 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:14.836014 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:14.836211 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:14.836285 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:14.838508 (+ 2223us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.87.131:55601: BlockingRecv error: recv got EOF from 127.31.87.131:55601 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260808 22:32:14.838738 14293 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 08 22:32:15 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.129@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:32:15 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.129@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:32:15.421259 14302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:15.416233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.129:43389)
0808 22:32:15.416515 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:15.416530 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:15.416625 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:15.416900 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:15.416903 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:15.417118 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:15.417228 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:15.417234 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:15.417940 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:15.417943 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:15.418507 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:15.418514 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:15.418867 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:15.418870 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:15.418926 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:15.419353 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:15.419363 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:15.421137 (+ 1774us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":93,"threads_started":1}
W20260808 22:32:15.421368 14303 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:15.416288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:43389 (local address 127.31.87.190:45463)
0808 22:32:15.416567 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:15.416571 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:15.416583 (+ 12us) server_negotiation.cc:407] Connection header received
0808 22:32:15.416716 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:15.416718 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:15.416772 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:15.416864 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:15.417325 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:15.417847 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:15.418588 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:15.418795 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:15.418858 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:15.421247 (+ 2389us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.87.129:43389: BlockingRecv error: recv got EOF from 127.31.87.129:43389 (error 108)
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":103,"threads_started":1}
Aug 08 22:32:15 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.130@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:32:15 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.130@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:32:15.626292 14305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:15.621519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.130:37883)
0808 22:32:15.621800 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:15.621812 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:15.621908 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:15.622111 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:15.622113 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:15.622300 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:15.622396 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:15.622402 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:15.623065 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:15.623067 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:15.623631 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:15.623638 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:15.623986 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:15.623988 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:15.624044 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:15.624483 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:15.624494 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:15.626162 (+ 1668us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":84,"threads_started":1}
W20260808 22:32:15.626395 14303 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:15.621603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:37883 (local address 127.31.87.190:45463)
0808 22:32:15.621738 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:15.621742 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:15.621871 (+ 129us) server_negotiation.cc:407] Connection header received
0808 22:32:15.621977 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:15.621979 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:15.622023 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:15.622088 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:15.622479 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:15.622988 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:15.623712 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:15.623912 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:15.623979 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:15.626298 (+ 2319us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.87.130:37883: BlockingRecv error: recv got EOF from 127.31.87.130:37883 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 08 22:32:15 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.131@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:32:15 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.131@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:32:15.845980 14307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:15.839384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.131:47195)
0808 22:32:15.839768 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:15.839783 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:15.839895 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:15.840195 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:15.840198 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:15.840420 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:15.840562 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:15.840571 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:15.841575 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:15.841579 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:15.842268 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:15.842280 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:15.842937 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:15.842941 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:15.843055 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:15.843607 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:15.843622 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:15.845827 (+ 2205us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":102,"threads_started":1}
W20260808 22:32:15.846100 14303 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:15.839429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:47195 (local address 127.31.87.190:45463)
0808 22:32:15.839582 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:15.839586 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:15.839868 (+ 282us) server_negotiation.cc:407] Connection header received
0808 22:32:15.840012 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:15.840015 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:15.840077 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:15.840167 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:15.840687 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:15.841467 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:15.842384 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:15.842692 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:15.842794 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:15.845978 (+ 3184us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.87.131:47195: BlockingRecv error: recv got EOF from 127.31.87.131:47195 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 08 22:32:16 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.129@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:32:16 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.129@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:32:16.426980 14308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:16.421896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.129:50007)
0808 22:32:16.422153 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:16.422166 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:16.422247 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:16.422454 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:16.422456 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:16.422624 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:16.422732 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:16.422738 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:16.423392 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:16.423395 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:16.423946 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:16.423952 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:16.424300 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:16.424302 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:16.424372 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:16.424810 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:16.424822 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:16.426839 (+ 2017us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":68,"threads_started":1}
W20260808 22:32:16.427114 14309 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:16.421972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:50007 (local address 127.31.87.190:45463)
0808 22:32:16.422210 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:16.422213 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:16.422225 (+ 12us) server_negotiation.cc:407] Connection header received
0808 22:32:16.422319 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:16.422321 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:16.422361 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:16.422431 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:16.422813 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:16.423320 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:16.424022 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:16.424227 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:16.424290 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:16.426985 (+ 2695us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.87.129:50007: BlockingRecv error: recv got EOF from 127.31.87.129:50007 (error 108)
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":74,"threads_started":1}
Aug 08 22:32:16 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.130@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:32:16 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.130@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:32:16.632130 14310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:16.627014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.130:39227)
0808 22:32:16.627271 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:16.627286 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:16.627382 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:16.627588 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:16.627590 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:16.627768 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:16.627863 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:16.627869 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:16.628633 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:16.628636 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:16.629469 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:16.629479 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:16.629873 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:16.629875 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:16.629931 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:16.630372 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:16.630382 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:16.631990 (+ 1608us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":74,"threads_started":1}
W20260808 22:32:16.632210 14309 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:16.627079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:39227 (local address 127.31.87.190:45463)
0808 22:32:16.627202 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:16.627206 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:16.627348 (+ 142us) server_negotiation.cc:407] Connection header received
0808 22:32:16.627449 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:16.627452 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:16.627496 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:16.627563 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:16.627947 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:16.628531 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:16.629570 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:16.629788 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:16.629865 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:16.632116 (+ 2251us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.87.130:39227: BlockingRecv error: recv got EOF from 127.31.87.130:39227 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 08 22:32:16 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.131@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:32:16 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.131@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:32:16.856383 14311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:16.846974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.131:56253)
0808 22:32:16.847228 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:16.847240 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:16.847336 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:16.847541 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:16.847543 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:16.847726 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:16.847859 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:16.847867 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:16.848555 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:16.848557 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:16.849135 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:16.849145 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:16.849507 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:16.849509 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:16.849566 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:16.854480 (+ 4914us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:16.854493 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:16.856250 (+ 1757us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":74,"threads_started":1}
W20260808 22:32:16.856515 14309 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:16.847091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:56253 (local address 127.31.87.190:45463)
0808 22:32:16.847213 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:16.847216 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:16.847289 (+ 73us) server_negotiation.cc:407] Connection header received
0808 22:32:16.847411 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:16.847413 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:16.847459 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:16.847515 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:16.847953 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:16.848476 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:16.849226 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:16.849433 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:16.849499 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:16.856396 (+ 6897us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.87.131:56253: BlockingRecv error: recv got EOF from 127.31.87.131:56253 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 08 22:32:17 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.129@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:32:17 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.129@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:32:17.432446 14312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:17.427682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.129:47921)
0808 22:32:17.427931 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:17.427944 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:17.428028 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:17.428230 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:17.428232 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:17.428403 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:17.428497 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:17.428502 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:17.429188 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:17.429190 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:17.429746 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:17.429755 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:17.430117 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:17.430120 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:17.430177 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:17.430599 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:17.430626 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:17.432305 (+ 1679us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":76,"threads_started":1}
W20260808 22:32:17.432546 14313 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:17.427758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:47921 (local address 127.31.87.190:45463)
0808 22:32:17.427996 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:17.427999 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:17.428012 (+ 13us) server_negotiation.cc:407] Connection header received
0808 22:32:17.428099 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:17.428102 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:17.428143 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:17.428216 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:17.428576 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:17.429082 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:17.429830 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:17.430041 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:17.430107 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:17.432438 (+ 2331us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.87.129:47921: BlockingRecv error: recv got EOF from 127.31.87.129:47921 (error 108)
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":77,"threads_started":1}
Aug 08 22:32:17 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.130@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:32:17 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.130@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:32:17.638060 14314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:17.632813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.130:40777)
0808 22:32:17.633083 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:17.633119 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:17.633223 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:17.633426 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:17.633428 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:17.633601 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:17.633696 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:17.633701 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:17.634439 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:17.634443 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:17.635021 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:17.635030 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:17.635393 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:17.635396 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:17.635458 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:17.635987 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:17.636002 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:17.637912 (+ 1910us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":100,"threads_started":1}
W20260808 22:32:17.638157 14313 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:17.632872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:40777 (local address 127.31.87.190:45463)
0808 22:32:17.633010 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:17.633015 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:17.633183 (+ 168us) server_negotiation.cc:407] Connection header received
0808 22:32:17.633292 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:17.633294 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:17.633340 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:17.633402 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:17.633773 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:17.634348 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:17.635109 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:17.635314 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:17.635383 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:17.638050 (+ 2667us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.87.130:40777: BlockingRecv error: recv got EOF from 127.31.87.130:40777 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 08 22:32:17 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.131@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:32:17 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.131@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:32:17.862269 14317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:17.857061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.131:59161)
0808 22:32:17.857369 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:17.857382 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:17.857491 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:17.857727 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:17.857731 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:17.857953 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:17.858054 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:17.858060 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:17.858803 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:17.858807 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:17.859394 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:17.859403 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:17.859831 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:17.859835 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:17.859920 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:17.860384 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:17.860396 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:17.862091 (+ 1695us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":87,"threads_started":1}
W20260808 22:32:17.862321 14313 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:17.857161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:59161 (local address 127.31.87.190:45463)
0808 22:32:17.857281 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:17.857287 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:17.857444 (+ 157us) server_negotiation.cc:407] Connection header received
0808 22:32:17.857572 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:17.857575 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:17.857630 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:17.857693 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:17.858160 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:17.858698 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:17.859501 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:17.859728 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:17.859814 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:17.862221 (+ 2407us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.87.131:59161: BlockingRecv error: recv got EOF from 127.31.87.131:59161 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 08 22:32:18 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.129@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:32:18 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.129@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:32:18.438714 14318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:18.433274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.129:40907)
0808 22:32:18.433532 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:18.433547 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:18.433662 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:18.433927 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:18.433929 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:18.434128 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:18.434237 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:18.434243 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:18.435020 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:18.435023 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:18.435615 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:18.435626 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:18.436091 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:18.436094 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:18.436175 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:18.436689 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:18.436703 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:18.438597 (+ 1894us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":73,"threads_started":1}
W20260808 22:32:18.438851 14319 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:18.433353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:40907 (local address 127.31.87.190:45463)
0808 22:32:18.433578 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:18.433581 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:18.433595 (+ 14us) server_negotiation.cc:407] Connection header received
0808 22:32:18.433778 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:18.433780 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:18.433826 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:18.433901 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:18.434326 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:18.434917 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:18.435710 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:18.435977 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:18.436063 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:18.438735 (+ 2672us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.87.129:40907: BlockingRecv error: recv got EOF from 127.31.87.129:40907 (error 108)
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":68,"threads_started":1}
Aug 08 22:32:18 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.130@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:32:18 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.130@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:32:18.643469 14320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:18.638755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.130:39021)
0808 22:32:18.639018 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:18.639031 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:18.639131 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:18.639334 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:18.639336 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:18.639509 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:18.639625 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:18.639631 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:18.640277 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:18.640279 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:18.640835 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:18.640841 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:18.641213 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:18.641215 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:18.641274 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:18.641709 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:18.641720 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:18.643357 (+ 1637us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":78,"threads_started":1}
W20260808 22:32:18.643584 14319 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:18.638815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:39021 (local address 127.31.87.190:45463)
0808 22:32:18.638962 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:18.638967 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:18.639091 (+ 124us) server_negotiation.cc:407] Connection header received
0808 22:32:18.639196 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:18.639198 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:18.639241 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:18.639308 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:18.639706 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:18.640202 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:18.640912 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:18.641133 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:18.641210 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:18.643483 (+ 2273us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.87.130:39021: BlockingRecv error: recv got EOF from 127.31.87.130:39021 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 08 22:32:18 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.131@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:32:18 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.131@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:32:18.868242 14322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:18.863104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.131:53889)
0808 22:32:18.863370 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:18.863383 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:18.863490 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:18.863743 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:18.863746 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:18.863943 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:18.864039 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:18.864045 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:18.864773 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:18.864777 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:18.865410 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:18.865420 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:18.865852 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:18.865855 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:18.865940 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:18.866428 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:18.866439 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:18.868091 (+ 1652us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":85,"threads_started":1}
W20260808 22:32:18.868337 14319 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:18.863136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:53889 (local address 127.31.87.190:45463)
0808 22:32:18.863270 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:18.863274 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:18.863456 (+ 182us) server_negotiation.cc:407] Connection header received
0808 22:32:18.863578 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:18.863582 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:18.863641 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:18.863705 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:18.864148 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:18.864671 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:18.865522 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:18.865748 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:18.865841 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:18.868235 (+ 2394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.87.131:53889: BlockingRecv error: recv got EOF from 127.31.87.131:53889 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 08 22:32:19 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.129@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:32:19 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.129@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:32:19.445489 14324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:19.439572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.129:60091)
0808 22:32:19.439874 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:19.439892 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:19.440013 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:19.440267 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:19.440271 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:19.440510 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:19.440640 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:19.440648 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:19.441407 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:19.441411 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:19.442324 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:19.442337 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:19.442762 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:19.442767 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:19.442860 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:19.443389 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:19.443405 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:19.445372 (+ 1967us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":79,"threads_started":1}
W20260808 22:32:19.445685 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:19.439640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:60091 (local address 127.31.87.190:45463)
0808 22:32:19.439877 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:19.439881 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:19.439958 (+ 77us) server_negotiation.cc:407] Connection header received
0808 22:32:19.440112 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:19.440114 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:19.440155 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:19.440229 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:19.440752 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:19.441304 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:19.442446 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:19.442664 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:19.442732 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:19.445541 (+ 2809us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.87.129:60091: BlockingRecv error: recv got EOF from 127.31.87.129:60091 (error 108)
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":57,"threads_started":1}
Aug 08 22:32:19 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.130@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:32:19 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.130@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:32:19.649510 14327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:19.644245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.130:35967)
0808 22:32:19.644512 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:19.644526 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:19.644616 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:19.644845 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:19.644848 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:19.645062 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:19.645208 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:19.645217 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:19.646111 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:19.646114 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:19.646690 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:19.646699 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:19.647088 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:19.647090 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:19.647146 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:19.647633 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:19.647647 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:19.649366 (+ 1719us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":81,"threads_started":1}
W20260808 22:32:19.649600 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:19.644359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:35967 (local address 127.31.87.190:45463)
0808 22:32:19.644496 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:19.644499 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:19.644585 (+ 86us) server_negotiation.cc:407] Connection header received
0808 22:32:19.644684 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:19.644686 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:19.644729 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:19.644792 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:19.645320 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:19.646019 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:19.646793 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:19.647010 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:19.647081 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:19.649501 (+ 2420us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.87.130:35967: BlockingRecv error: recv got EOF from 127.31.87.130:35967 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 08 22:32:19 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.131@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:32:19 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.131@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:32:19.873960 14328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:19.869071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.131:37493)
0808 22:32:19.869340 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:19.869354 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:19.869444 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:19.869652 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:19.869654 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:19.869844 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:19.869941 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:19.869947 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:19.870652 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:19.870654 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:19.871201 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:19.871207 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:19.871583 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:19.871586 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:19.871641 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:19.872137 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:19.872149 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:19.873843 (+ 1694us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":68,"threads_started":1}
W20260808 22:32:19.874119 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:19.869160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:37493 (local address 127.31.87.190:45463)
0808 22:32:19.869292 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:19.869296 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:19.869404 (+ 108us) server_negotiation.cc:407] Connection header received
0808 22:32:19.869512 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:19.869514 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:19.869557 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:19.869631 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:19.870028 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:19.870574 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:19.871285 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:19.871496 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:19.871576 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:19.873992 (+ 2416us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.87.131:37493: BlockingRecv error: recv got EOF from 127.31.87.131:37493 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 08 22:32:20 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.129@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:32:20 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.129@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:32:20.451712 14330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:20.446376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.129:38141)
0808 22:32:20.446717 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:20.446733 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:20.446853 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:20.447127 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:20.447129 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:20.447329 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:20.447454 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:20.447463 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:20.448355 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:20.448357 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:20.448907 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:20.448915 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:20.449352 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:20.449355 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:20.449413 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:20.449864 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:20.449875 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:20.451567 (+ 1692us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":91,"threads_started":1}
W20260808 22:32:20.451828 14331 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:20.446439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:38141 (local address 127.31.87.190:45463)
0808 22:32:20.446718 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:20.446723 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:20.446792 (+ 69us) server_negotiation.cc:407] Connection header received
0808 22:32:20.446954 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:20.446957 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:20.447011 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:20.447094 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:20.447576 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:20.448239 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:20.448995 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:20.449272 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:20.449346 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:20.451700 (+ 2354us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.87.129:38141: BlockingRecv error: recv got EOF from 127.31.87.129:38141 (error 108)
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":75,"threads_started":1}
Aug 08 22:32:20 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.130@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:32:20 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.130@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:32:20.655673 14332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:20.650264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.130:51021)
0808 22:32:20.650539 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:20.650551 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:20.650655 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:20.650903 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:20.650906 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:20.651109 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:20.651201 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:20.651207 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:20.651932 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:20.651935 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:20.652488 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:20.652496 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:20.652930 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:20.652933 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:20.653019 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:20.653526 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:20.653540 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:20.655533 (+ 1993us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":93,"threads_started":1}
W20260808 22:32:20.655774 14331 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:20.650332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:51021 (local address 127.31.87.190:45463)
0808 22:32:20.650464 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:20.650469 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:20.650616 (+ 147us) server_negotiation.cc:407] Connection header received
0808 22:32:20.650740 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:20.650743 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:20.650799 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:20.650865 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:20.651312 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:20.651832 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:20.652598 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:20.652821 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:20.652905 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:20.655659 (+ 2754us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.87.130:51021: BlockingRecv error: recv got EOF from 127.31.87.130:51021 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 08 22:32:20 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.131@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:32:20 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.131@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:32:20.879437 14333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:20.874685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.131:39795)
0808 22:32:20.874940 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:20.874953 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:20.875051 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:20.875241 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:20.875243 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:20.875422 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:20.875519 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:20.875524 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:20.876182 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:20.876185 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:20.876736 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:20.876743 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:20.877127 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:20.877132 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:20.877194 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:20.877628 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:20.877639 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:20.879320 (+ 1681us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":75,"threads_started":1}
W20260808 22:32:20.879560 14331 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:20.874733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:39795 (local address 127.31.87.190:45463)
0808 22:32:20.874852 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:20.874855 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:20.875016 (+ 161us) server_negotiation.cc:407] Connection header received
0808 22:32:20.875118 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:20.875120 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:20.875166 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:20.875226 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:20.875597 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:20.876108 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:20.876814 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:20.877014 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:20.877077 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:20.879442 (+ 2365us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.87.131:39795: BlockingRecv error: recv got EOF from 127.31.87.131:39795 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 08 22:32:21 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.129@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:32:21 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.129@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:32:21.458042 14335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:21.452533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.129:36837)
0808 22:32:21.452865 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:21.452881 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:21.453001 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:21.453276 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:21.453280 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:21.453483 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:21.453582 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:21.453590 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:21.454425 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:21.454429 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:21.455035 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:21.455046 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:21.455502 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:21.455506 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:21.455589 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:21.456033 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:21.456045 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:21.457926 (+ 1881us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":81,"threads_started":1}
W20260808 22:32:21.458174 14337 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:21.452617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:36837 (local address 127.31.87.190:45463)
0808 22:32:21.452875 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:21.452880 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:21.452954 (+ 74us) server_negotiation.cc:407] Connection header received
0808 22:32:21.453108 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:21.453112 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:21.453163 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:21.453241 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:21.453713 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:21.454321 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:21.455159 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:21.455399 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:21.455487 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:21.458055 (+ 2568us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.87.129:36837: BlockingRecv error: recv got EOF from 127.31.87.129:36837 (error 108)
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":80,"threads_started":1}
Aug 08 22:32:21 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.130@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:32:21 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.130@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:32:21.661743 14338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:21.656374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.130:59251)
0808 22:32:21.656728 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:21.656744 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:21.656862 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:21.657129 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:21.657132 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:21.657344 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:21.657436 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:21.657443 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:21.658227 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:21.658231 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:21.658866 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:21.658878 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:21.659305 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:21.659309 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:21.659391 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:21.659848 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:21.659859 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:21.661595 (+ 1736us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":84,"threads_started":1}
W20260808 22:32:21.661831 14337 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:21.656461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:59251 (local address 127.31.87.190:45463)
0808 22:32:21.656615 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:21.656620 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:21.656818 (+ 198us) server_negotiation.cc:407] Connection header received
0808 22:32:21.656948 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:21.656951 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:21.657001 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:21.657065 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:21.657549 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:21.658127 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:21.658973 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:21.659203 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:21.659291 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:21.661733 (+ 2442us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.87.130:59251: BlockingRecv error: recv got EOF from 127.31.87.130:59251 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 08 22:32:21 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.131@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:32:21 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.131@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:32:21.888996 14340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:21.880178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.131:54871)
0808 22:32:21.880521 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:21.880535 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:21.880666 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:21.880943 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:21.880946 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:21.881200 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:21.881307 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:21.881314 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:21.882109 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:21.882114 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:21.882694 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:21.882704 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:21.883162 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:21.883167 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:21.883259 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:21.887185 (+ 3926us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:21.887198 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:21.888845 (+ 1647us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":84,"threads_started":1}
W20260808 22:32:21.889106 14337 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:21.880243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:54871 (local address 127.31.87.190:45463)
0808 22:32:21.880387 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:21.880393 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:21.880625 (+ 232us) server_negotiation.cc:407] Connection header received
0808 22:32:21.880766 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:21.880770 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:21.880831 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:21.880907 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:21.881454 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:21.881994 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:21.882820 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:21.883044 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:21.883148 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:21.888983 (+ 5835us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.87.131:54871: BlockingRecv error: recv got EOF from 127.31.87.131:54871 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 08 22:32:22 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.129@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:32:22 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.129@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:32:22.463716 14341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:22.458699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.129:47193)
0808 22:32:22.458959 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:22.458972 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:22.459060 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:22.459268 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:22.459270 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:22.459448 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:22.459542 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:22.459547 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:22.460300 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:22.460304 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:22.460872 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:22.460881 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:22.461300 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:22.461304 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:22.461367 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:22.461802 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:22.461813 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:22.463545 (+ 1732us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":89,"threads_started":1}
W20260808 22:32:22.463804 14342 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:22.458841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:47193 (local address 127.31.87.190:45463)
0808 22:32:22.459048 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:22.459052 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:22.459064 (+ 12us) server_negotiation.cc:407] Connection header received
0808 22:32:22.459139 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:22.459141 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:22.459184 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:22.459247 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:22.459623 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:22.460203 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:22.460959 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:22.461210 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:22.461287 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:22.463687 (+ 2400us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.87.129:47193: BlockingRecv error: recv got EOF from 127.31.87.129:47193 (error 108)
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":78,"threads_started":1}
Aug 08 22:32:22 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.130@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:32:22 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.130@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:32:22.667559 14343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:22.662529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.130:39473)
0808 22:32:22.662782 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:22.662795 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:22.662893 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:22.663097 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:22.663099 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:22.663288 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:22.663383 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:22.663389 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:22.664196 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:22.664198 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:22.664745 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:22.664752 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:22.665137 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:22.665141 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:22.665205 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:22.665659 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:22.665670 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:22.667410 (+ 1740us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":80,"threads_started":1}
W20260808 22:32:22.667622 14342 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:22.662596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:39473 (local address 127.31.87.190:45463)
0808 22:32:22.662727 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:22.662731 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:22.662855 (+ 124us) server_negotiation.cc:407] Connection header received
0808 22:32:22.662962 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:22.662965 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:22.663008 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:22.663073 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:22.663465 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:22.664114 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:22.664824 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:22.665026 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:22.665121 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:22.667526 (+ 2405us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.87.130:39473: BlockingRecv error: recv got EOF from 127.31.87.130:39473 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 08 22:32:22 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.131@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:32:22 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.131@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:32:22.895022 14344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:22.889853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.131:58911)
0808 22:32:22.890163 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:22.890176 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:22.890286 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:22.890533 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:22.890536 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:22.890732 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:22.890839 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:22.890845 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:22.891566 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:22.891570 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:22.892122 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:22.892130 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:22.892566 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:22.892570 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:22.892653 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:22.893163 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:22.893175 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:22.894869 (+ 1694us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":129,"threads_started":1}
W20260808 22:32:22.895105 14342 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:22.889936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:58911 (local address 127.31.87.190:45463)
0808 22:32:22.890092 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:22.890096 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:22.890246 (+ 150us) server_negotiation.cc:407] Connection header received
0808 22:32:22.890380 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:22.890383 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:22.890430 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:22.890494 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:22.890954 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:22.891467 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:22.892245 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:22.892467 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:22.892554 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:22.895003 (+ 2449us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.87.131:58911: BlockingRecv error: recv got EOF from 127.31.87.131:58911 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 08 22:32:23 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.129@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:32:23 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.129@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:32:23.469630 14346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:23.464451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.129:37815)
0808 22:32:23.464758 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:23.464774 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:23.464891 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:23.465162 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:23.465165 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:23.465366 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:23.465461 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:23.465468 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:23.466214 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:23.466218 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:23.466760 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:23.466769 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:23.467197 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:23.467202 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:23.467283 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:23.467724 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:23.467735 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:23.469469 (+ 1734us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":91,"threads_started":1}
W20260808 22:32:23.469758 14347 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:23.464523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:37815 (local address 127.31.87.190:45463)
0808 22:32:23.464759 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:23.464764 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:23.464843 (+ 79us) server_negotiation.cc:407] Connection header received
0808 22:32:23.464975 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:23.464978 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:23.465025 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:23.465135 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:23.465579 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:23.466110 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:23.466872 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:23.467098 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:23.467182 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:23.469640 (+ 2458us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.87.129:37815: BlockingRecv error: recv got EOF from 127.31.87.129:37815 (error 108)
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":56,"threads_started":1}
Aug 08 22:32:23 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.130@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:32:23 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.130@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:32:23.673300 14348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:23.668390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.130:54371)
0808 22:32:23.668679 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:23.668693 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:23.668795 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:23.668994 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:23.668996 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:23.669198 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:23.669295 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:23.669300 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:23.669963 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:23.669965 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:23.670511 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:23.670517 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:23.670876 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:23.670878 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:23.670935 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:23.671399 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:23.671411 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:23.673159 (+ 1748us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":99,"threads_started":1}
W20260808 22:32:23.673368 14347 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:23.668473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:54371 (local address 127.31.87.190:45463)
0808 22:32:23.668602 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:23.668605 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:23.668754 (+ 149us) server_negotiation.cc:407] Connection header received
0808 22:32:23.668857 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:23.668859 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:23.668903 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:23.668966 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:23.669376 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:23.669886 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:23.670589 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:23.670801 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:23.670866 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:23.673275 (+ 2409us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.87.130:54371: BlockingRecv error: recv got EOF from 127.31.87.130:54371 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 08 22:32:23 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.131@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:32:23 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.131@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:32:23.900799 14350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:23.895743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.131:53437)
0808 22:32:23.896035 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:23.896048 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:23.896157 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:23.896364 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:23.896366 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:23.896542 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:23.896642 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:23.896648 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:23.897328 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:23.897330 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:23.897868 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:23.897875 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:23.898224 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:23.898226 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:23.898282 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:23.898801 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:23.898812 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:23.900647 (+ 1835us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":110,"threads_started":1}
W20260808 22:32:23.900872 14347 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:23.895815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:53437 (local address 127.31.87.190:45463)
0808 22:32:23.895941 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:23.895944 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:23.896106 (+ 162us) server_negotiation.cc:407] Connection header received
0808 22:32:23.896221 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:23.896223 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:23.896269 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:23.896345 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:23.896723 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:23.897257 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:23.897948 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:23.898148 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:23.898215 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:23.900772 (+ 2557us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.87.131:53437: BlockingRecv error: recv got EOF from 127.31.87.131:53437 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 08 22:32:24 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.129@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:32:24 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.129@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:32:24.475579 14351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:24.470397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.129:48239)
0808 22:32:24.470647 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:24.470662 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:24.470746 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:24.470969 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:24.470971 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:24.471149 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:24.471244 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:24.471249 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:24.471974 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:24.471978 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:24.472664 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:24.472675 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:24.473214 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:24.473217 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:24.473279 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:24.473755 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:24.473781 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:24.475465 (+ 1684us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":69,"threads_started":1}
W20260808 22:32:24.475723 14352 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:24.470471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:48239 (local address 127.31.87.190:45463)
0808 22:32:24.470730 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:24.470733 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:24.470747 (+ 14us) server_negotiation.cc:407] Connection header received
0808 22:32:24.470830 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:24.470832 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:24.470872 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:24.470947 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:24.471327 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:24.471879 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:24.472784 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:24.473074 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:24.473202 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:24.475596 (+ 2394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.87.129:48239: BlockingRecv error: recv got EOF from 127.31.87.129:48239 (error 108)
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":85,"threads_started":1}
Aug 08 22:32:24 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.130@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:32:24 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.130@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:32:24.679023 14354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:24.674186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.130:33955)
0808 22:32:24.674438 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:24.674451 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:24.674540 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:24.674744 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:24.674746 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:24.674924 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:24.675016 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:24.675021 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:24.675705 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:24.675707 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:24.676253 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:24.676259 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:24.676628 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:24.676630 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:24.676688 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:24.677183 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:24.677194 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:24.678907 (+ 1713us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":89,"threads_started":1}
W20260808 22:32:24.679164 14352 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:24.674200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:33955 (local address 127.31.87.190:45463)
0808 22:32:24.674336 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:24.674341 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:24.674518 (+ 177us) server_negotiation.cc:407] Connection header received
0808 22:32:24.674611 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:24.674613 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:24.674663 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:24.674737 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:24.675105 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:24.675628 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:24.676342 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:24.676550 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:24.676627 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:24.679036 (+ 2409us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.87.130:33955: BlockingRecv error: recv got EOF from 127.31.87.130:33955 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 08 22:32:24 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.131@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:32:24 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.131@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:32:24.906877 14355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:24.901669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.131:34601)
0808 22:32:24.901953 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:24.901968 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:24.902082 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:24.902296 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:24.902298 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:24.902498 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:24.902614 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:24.902621 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:24.903303 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:24.903306 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:24.903871 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:24.903878 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:24.904233 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:24.904236 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:24.904296 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:24.904850 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:24.904864 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:24.906735 (+ 1871us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":84,"threads_started":1}
W20260808 22:32:24.906986 14352 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:24.901780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:34601 (local address 127.31.87.190:45463)
0808 22:32:24.901900 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:24.901903 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:24.902031 (+ 128us) server_negotiation.cc:407] Connection header received
0808 22:32:24.902155 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:24.902157 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:24.902199 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:24.902263 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:24.902701 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:24.903215 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:24.903954 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:24.904155 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:24.904219 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:24.906879 (+ 2660us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.87.131:34601: BlockingRecv error: recv got EOF from 127.31.87.131:34601 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 08 22:32:25 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.129@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:32:25 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.129@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:32:25.481472 14356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:25.476327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.129:38423)
0808 22:32:25.476614 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:25.476628 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:25.476724 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:25.476948 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:25.476950 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:25.477187 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:25.477292 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:25.477299 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:25.478011 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:25.478014 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:25.478612 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:25.478622 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:25.479011 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:25.479013 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:25.479089 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:25.479590 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:25.479602 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:25.481353 (+ 1751us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":94,"threads_started":1}
W20260808 22:32:25.481602 14358 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:25.476389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:38423 (local address 127.31.87.190:45463)
0808 22:32:25.476607 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:25.476610 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:25.476686 (+ 76us) server_negotiation.cc:407] Connection header received
0808 22:32:25.476798 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:25.476801 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:25.476842 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:25.476913 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:25.477379 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:25.477897 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:25.478702 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:25.478929 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:25.479001 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:25.481473 (+ 2472us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.87.129:38423: BlockingRecv error: recv got EOF from 127.31.87.129:38423 (error 108)
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":69,"threads_started":1}
Aug 08 22:32:25 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.130@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:32:25 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.130@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:32:25.684890 14360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:25.679730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.130:59185)
0808 22:32:25.680011 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:25.680024 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:25.680128 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:25.680353 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:25.680356 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:25.680538 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:25.680660 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:25.680666 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:25.681353 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:25.681355 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:25.681908 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:25.681915 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:25.682311 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:25.682313 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:25.682368 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:25.682856 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:25.682868 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:25.684737 (+ 1869us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":92,"threads_started":1}
W20260808 22:32:25.684952 14358 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:25.679782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:59185 (local address 127.31.87.190:45463)
0808 22:32:25.679906 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:25.679910 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:25.680084 (+ 174us) server_negotiation.cc:407] Connection header received
0808 22:32:25.680192 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:25.680194 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:25.680254 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:25.680317 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:25.680748 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:25.681283 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:25.682006 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:25.682230 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:25.682303 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:25.684855 (+ 2552us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.87.130:59185: BlockingRecv error: recv got EOF from 127.31.87.130:59185 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 08 22:32:25 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.131@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:32:25 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.131@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:32:25.912606 14361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:25.907558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.131:58821)
0808 22:32:25.907816 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:25.907830 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:25.907926 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:25.908133 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:25.908135 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:25.908312 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:25.908412 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:25.908417 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:25.909121 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:25.909124 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:25.909662 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:25.909670 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:25.910049 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:25.910051 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:25.910108 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:25.910572 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:25.910583 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:25.912451 (+ 1868us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":72,"threads_started":1}
W20260808 22:32:25.912693 14358 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:25.907623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:58821 (local address 127.31.87.190:45463)
0808 22:32:25.907744 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:25.907747 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:25.907889 (+ 142us) server_negotiation.cc:407] Connection header received
0808 22:32:25.908002 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:25.908004 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:25.908049 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:25.908105 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:25.908489 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:25.909018 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:25.909753 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:25.909971 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:25.910039 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:25.912565 (+ 2526us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.87.131:58821: BlockingRecv error: recv got EOF from 127.31.87.131:58821 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 08 22:32:26 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.129@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:32:26 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.129@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:32:26.487605 14363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:26.482286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.129:57533)
0808 22:32:26.482553 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:26.482567 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:26.482668 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:26.482903 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:26.482905 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:26.483100 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:26.483208 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:26.483215 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:26.483910 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:26.483912 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:26.484487 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:26.484497 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:26.484886 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:26.484888 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:26.484943 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:26.485455 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:26.485467 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:26.487459 (+ 1992us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":86,"threads_started":1}
W20260808 22:32:26.487701 14364 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:26.482362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:57533 (local address 127.31.87.190:45463)
0808 22:32:26.482577 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:26.482581 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:26.482635 (+ 54us) server_negotiation.cc:407] Connection header received
0808 22:32:26.482740 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:26.482743 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:26.482790 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:26.482865 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:26.483310 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:26.483827 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:26.484583 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:26.484804 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:26.484877 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:26.487589 (+ 2712us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.87.129:57533: BlockingRecv error: recv got EOF from 127.31.87.129:57533 (error 108)
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":59,"threads_started":1}
Aug 08 22:32:26 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.130@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:32:26 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.130@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:32:26.690908 14366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:26.685666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.130:50009)
0808 22:32:26.685959 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:26.685972 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:26.686070 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:26.686346 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:26.686349 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:26.686573 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:26.686703 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:26.686712 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:26.687634 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:26.687637 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:26.688191 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:26.688200 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:26.688569 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:26.688571 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:26.688628 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:26.689082 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:26.689117 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:26.690792 (+ 1675us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":86,"threads_started":1}
W20260808 22:32:26.691015 14364 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:26.685743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:50009 (local address 127.31.87.190:45463)
0808 22:32:26.685891 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:26.685895 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:26.686030 (+ 135us) server_negotiation.cc:407] Connection header received
0808 22:32:26.686161 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:26.686164 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:26.686221 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:26.686292 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:26.686829 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:26.687530 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:26.688283 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:26.688493 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:26.688563 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:26.690916 (+ 2353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.87.130:50009: BlockingRecv error: recv got EOF from 127.31.87.130:50009 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 08 22:32:26 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.131@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:32:26 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.131@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:32:26.921180 14367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:26.913357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.131:41371)
0808 22:32:26.913631 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:26.913644 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:26.913753 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:26.913998 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:26.914001 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:26.914207 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:26.914330 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:26.914336 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:26.915076 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:26.915080 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:26.915637 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:26.915645 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:26.916082 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:26.916086 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:26.916174 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:26.919286 (+ 3112us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:26.919297 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:26.920998 (+ 1701us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":87,"threads_started":1}
W20260808 22:32:26.921279 14364 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:26.913430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:41371 (local address 127.31.87.190:45463)
0808 22:32:26.913576 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:26.913581 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:26.913709 (+ 128us) server_negotiation.cc:407] Connection header received
0808 22:32:26.913842 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:26.913846 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:26.913900 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:26.913963 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:26.914441 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:26.914975 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:26.915749 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:26.915982 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:26.916067 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:26.921176 (+ 5109us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.87.131:41371: BlockingRecv error: recv got EOF from 127.31.87.131:41371 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 08 22:32:27 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.129@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:32:27 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.129@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:32:27.493257 14369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:27.488352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.129:49007)
0808 22:32:27.488637 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:27.488651 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:27.488750 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:27.488959 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:27.488961 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:27.489165 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:27.489264 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:27.489275 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:27.489957 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:27.489959 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:27.490504 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:27.490511 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:27.490863 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:27.490866 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:27.490921 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:27.491359 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:27.491370 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:27.493128 (+ 1758us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":86,"threads_started":1}
W20260808 22:32:27.493364 14370 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:27.488413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:49007 (local address 127.31.87.190:45463)
0808 22:32:27.488649 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:27.488652 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:27.488712 (+ 60us) server_negotiation.cc:407] Connection header received
0808 22:32:27.488818 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:27.488820 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:27.488863 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:27.488935 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:27.489350 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:27.489875 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:27.490580 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:27.490791 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:27.490856 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:27.493255 (+ 2399us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.87.129:49007: BlockingRecv error: recv got EOF from 127.31.87.129:49007 (error 108)
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":80,"threads_started":1}
Aug 08 22:32:27 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.130@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:32:27 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.130@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:32:27.696890 14371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:27.691648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.130:40871)
0808 22:32:27.691918 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:27.691933 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:27.692043 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:27.692285 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:27.692288 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:27.692488 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:27.692580 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:27.692586 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:27.693356 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:27.693360 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:27.693921 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:27.693929 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:27.694336 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:27.694340 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:27.694419 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:27.694893 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:27.694905 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:27.696770 (+ 1865us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":85,"threads_started":1}
W20260808 22:32:27.696985 14370 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:27.691790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:40871 (local address 127.31.87.190:45463)
0808 22:32:27.691919 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:27.691924 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:27.691995 (+ 71us) server_negotiation.cc:407] Connection header received
0808 22:32:27.692125 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:27.692128 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:27.692171 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:27.692238 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:27.692689 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:27.693249 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:27.694025 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:27.694239 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:27.694326 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:27.696886 (+ 2560us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.87.130:40871: BlockingRecv error: recv got EOF from 127.31.87.130:40871 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 08 22:32:27 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.131@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:32:27 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.131@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:32:27.926851 14372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:27.922038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.131:52305)
0808 22:32:27.922305 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:27.922318 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:27.922413 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:27.922613 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:27.922615 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:27.922805 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:27.922901 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:27.922907 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:27.923569 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:27.923571 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:27.924118 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:27.924125 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:27.924481 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:27.924484 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:27.924539 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:27.924995 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:27.925006 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:27.926725 (+ 1719us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":86,"threads_started":1}
W20260808 22:32:27.926985 14370 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:27.922099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:52305 (local address 127.31.87.190:45463)
0808 22:32:27.922223 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:27.922227 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:27.922377 (+ 150us) server_negotiation.cc:407] Connection header received
0808 22:32:27.922483 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:27.922486 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:27.922530 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:27.922602 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:27.922981 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:27.923493 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:27.924198 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:27.924403 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:27.924471 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:27.926874 (+ 2403us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.87.131:52305: BlockingRecv error: recv got EOF from 127.31.87.131:52305 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 08 22:32:28 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.129@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:32:28 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.129@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:32:28.499401 14373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:28.494010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.129:51929)
0808 22:32:28.494298 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:28.494315 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:28.494414 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:28.494621 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:28.494623 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:28.494817 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:28.494939 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:28.494949 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:28.495915 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:28.495918 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:28.496506 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:28.496515 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:28.496905 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:28.496908 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:28.496963 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:28.497476 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:28.497488 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:28.499242 (+ 1754us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":78,"threads_started":1}
W20260808 22:32:28.499573 14374 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:28.494083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:51929 (local address 127.31.87.190:45463)
0808 22:32:28.494408 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:28.494412 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:28.494425 (+ 13us) server_negotiation.cc:407] Connection header received
0808 22:32:28.494481 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:28.494483 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:28.494525 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:28.494596 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:28.495054 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:28.495807 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:28.496599 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:28.496825 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:28.496896 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:28.499369 (+ 2473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.87.129:51929: BlockingRecv error: recv got EOF from 127.31.87.129:51929 (error 108)
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":74,"threads_started":1}
Aug 08 22:32:28 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.130@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:32:28 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.130@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:32:28.702775 14375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:28.697578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.130:45171)
0808 22:32:28.697829 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:28.697844 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:28.697952 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:28.698202 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:28.698205 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:28.698413 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:28.698513 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:28.698520 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:28.699271 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:28.699275 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:28.699857 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:28.699866 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:28.700293 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:28.700297 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:28.700380 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:28.700881 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:28.700893 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:28.702647 (+ 1754us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":78,"threads_started":1}
W20260808 22:32:28.702873 14374 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:28.697622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:45171 (local address 127.31.87.190:45463)
0808 22:32:28.697776 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:28.697781 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:28.697911 (+ 130us) server_negotiation.cc:407] Connection header received
0808 22:32:28.698039 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:28.698042 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:28.698090 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:28.698162 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:28.698629 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:28.699165 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:28.699971 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:28.700196 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:28.700282 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:28.702776 (+ 2494us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.87.130:45171: BlockingRecv error: recv got EOF from 127.31.87.130:45171 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 08 22:32:28 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.131@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:32:28 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.131@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:32:28.932900 14378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:28.927561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.131:33639)
0808 22:32:28.927847 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:28.927860 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:28.927957 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:28.928162 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:28.928164 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:28.928347 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:28.928446 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:28.928452 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:28.929207 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:28.929210 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:28.929794 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:28.929804 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:28.930189 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:28.930192 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:28.930252 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:28.930803 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:28.930816 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:28.932751 (+ 1935us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":104,"threads_started":1}
W20260808 22:32:28.933017 14374 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:28.927661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:33639 (local address 127.31.87.190:45463)
0808 22:32:28.927791 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:28.927794 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:28.927919 (+ 125us) server_negotiation.cc:407] Connection header received
0808 22:32:28.928019 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:28.928021 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:28.928066 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:28.928129 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:28.928528 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:28.929028 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:28.929893 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:28.930103 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:28.930171 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:28.932898 (+ 2727us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.87.131:33639: BlockingRecv error: recv got EOF from 127.31.87.131:33639 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 08 22:32:29 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.129@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:32:29 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.129@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:32:29.505177 14379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:29.500251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.129:39725)
0808 22:32:29.500522 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:29.500536 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:29.500620 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:29.500824 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:29.500826 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:29.500998 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:29.501119 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:29.501127 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:29.501837 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:29.501839 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:29.502390 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:29.502397 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:29.502773 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:29.502775 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:29.502830 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:29.503257 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:29.503278 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:29.505014 (+ 1736us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":87,"threads_started":1}
W20260808 22:32:29.505321 14380 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:29.500317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:39725 (local address 127.31.87.190:45463)
0808 22:32:29.500588 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:29.500591 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:29.500606 (+ 15us) server_negotiation.cc:407] Connection header received
0808 22:32:29.500697 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:29.500699 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:29.500740 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:29.500813 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:29.501219 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:29.501760 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:29.502467 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:29.502698 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:29.502766 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:29.505185 (+ 2419us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.87.129:39725: BlockingRecv error: recv got EOF from 127.31.87.129:39725 (error 108)
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":78,"threads_started":1}
Aug 08 22:32:29 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.130@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:32:29 dist-test-slave-266d krb5kdc[12707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.87.130@KRBTEST.COM for oryx/127.31.87.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:32:29.709034 14383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:29.703655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45463 (local address 127.31.87.130:56459)
0808 22:32:29.703923 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:29.703936 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:29.704030 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:29.704237 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:29.704239 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:29.704422 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:29.704518 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:29.704523 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:29.705205 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:29.705207 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:29.705803 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:29.705813 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:29.706218 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:29.706222 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:29.706322 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:29.706916 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:29.706935 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:29.708891 (+ 1956us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.87.190:45463: Server oryx/127.31.87.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":107,"threads_started":1}
W20260808 22:32:29.709142 14380 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:29.703707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:56459 (local address 127.31.87.190:45463)
0808 22:32:29.703852 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:29.703855 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:29.703993 (+ 138us) server_negotiation.cc:407] Connection header received
0808 22:32:29.704099 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:29.704102 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:29.704144 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:29.704206 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:29.704599 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:29.705135 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:29.705889 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:29.706092 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:29.706156 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:29.709015 (+ 2859us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.87.130:56459: BlockingRecv error: recv got EOF from 127.31.87.130:56459 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 22:32:29.830585 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 13653
I20260808 22:32:29.848615 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 13897
I20260808 22:32:29.858515 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 14088
I20260808 22:32:29.869390 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 13532
2026-08-08T22:32:29Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18110 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-08T22:32:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:32:29Z Disabled control of system clock
I20260808 22:32:29.906106 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:39615
--webserver_interface=127.31.87.190
--webserver_port=0
--builtin_ntp_servers=127.31.87.148:43401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:39615
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 22:32:30.014093 14389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:32:30.014335 14389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:32:30.014367 14389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:32:30.018199 14389 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:32:30.018244 14389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:32:30.018260 14389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:32:30.018275 14389 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:32:30.018287 14389 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:32:30.022842 14389 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:43401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:39615
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:39615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.87.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:32:30.023756 14389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:32:30.025009 14389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 22:32:30.033403 14389 server_base.cc:1034] running on GCE node
W20260808 22:32:30.033416 14396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:30.033416 14397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:30.033389 14400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:30.034034 14389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:32:30.035070 14389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:32:30.036224 14389 hybrid_clock.cc:648] HybridClock initialized: now 1786228350036203 us; error 31 us; skew 500 ppm
I20260808 22:32:30.037990 14389 webserver.cc:460] Webserver started at http://127.31.87.190:39971/ using document root <none> and password file <none>
I20260808 22:32:30.038542 14389 fs_manager.cc:359] Metadata directory not provided
I20260808 22:32:30.038604 14389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:32:30.038806 14389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:32:30.040850 14389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/master-0/data/instance:
uuid: "6e30354d85da48b398f1b740e6b6e01b"
format_stamp: "Formatted at 2026-08-08 22:32:30 on dist-test-slave-266d"
server_key: "d3447785e157067f03a7897428b27be0"
server_key_iv: "807f9fe48988cec45e03c3bba058a11e"
server_key_version: "encryptionkey@0"
I20260808 22:32:30.041414 14389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/master-0/wal/instance:
uuid: "6e30354d85da48b398f1b740e6b6e01b"
format_stamp: "Formatted at 2026-08-08 22:32:30 on dist-test-slave-266d"
server_key: "d3447785e157067f03a7897428b27be0"
server_key_iv: "807f9fe48988cec45e03c3bba058a11e"
server_key_version: "encryptionkey@0"
I20260808 22:32:30.044847 14389 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 22:32:30.047107 14412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:30.048226 14389 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 22:32:30.048329 14389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "6e30354d85da48b398f1b740e6b6e01b"
format_stamp: "Formatted at 2026-08-08 22:32:30 on dist-test-slave-266d"
server_key: "d3447785e157067f03a7897428b27be0"
server_key_iv: "807f9fe48988cec45e03c3bba058a11e"
server_key_version: "encryptionkey@0"
I20260808 22:32:30.048430 14389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:32:30.074343 14389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:32:30.074968 14389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:32:30.075150 14389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:32:30.081646 14389 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.190:39615
I20260808 22:32:30.081650 14491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.190:39615 every 8 connection(s)
I20260808 22:32:30.082532 14389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:32:30.085600 14492 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:32:30.090807 14492 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e30354d85da48b398f1b740e6b6e01b: Bootstrap starting.
I20260808 22:32:30.091223 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 14389
I20260808 22:32:30.091333 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/master-0/wal/instance
I20260808 22:32:30.091583 32094 external_mini_cluster.cc:1428] Setting key f96e5dafcb7d2c55298da35e029851ca
I20260808 22:32:30.093401 14492 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e30354d85da48b398f1b740e6b6e01b: Neither blocks nor log segments found. Creating new log.
I20260808 22:32:30.094094 14492 log.cc:824] T 00000000000000000000000000000000 P 6e30354d85da48b398f1b740e6b6e01b: Log is configured to *not* fsync() on all Append() calls
I20260808 22:32:30.095800 14492 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e30354d85da48b398f1b740e6b6e01b: No bootstrap required, opened a new log
I20260808 22:32:30.099244 14492 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e30354d85da48b398f1b740e6b6e01b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e30354d85da48b398f1b740e6b6e01b" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 39615 } }
I20260808 22:32:30.099483 14492 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e30354d85da48b398f1b740e6b6e01b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:32:30.099545 14492 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e30354d85da48b398f1b740e6b6e01b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e30354d85da48b398f1b740e6b6e01b, State: Initialized, Role: FOLLOWER
I20260808 22:32:30.099964 14492 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e30354d85da48b398f1b740e6b6e01b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e30354d85da48b398f1b740e6b6e01b" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 39615 } }
I20260808 22:32:30.100092 14492 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e30354d85da48b398f1b740e6b6e01b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:32:30.100155 14492 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e30354d85da48b398f1b740e6b6e01b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:32:30.100250 14492 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e30354d85da48b398f1b740e6b6e01b [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:32:30.100329 14496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:30.092857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51694 (local address 127.31.87.190:39615)
0808 22:32:30.093487 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:30.093506 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:30.093549 (+ 43us) server_negotiation.cc:407] Connection header received
0808 22:32:30.094151 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:30.094167 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:30.094561 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:30.094856 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:30.095261 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:30.096236 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:30.097160 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:30.097644 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:30.097785 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:30.098611 (+ 826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:30.098617 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:30.098631 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:32:30.098992 (+ 361us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:30.099045 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:32:30.099065 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:30.099153 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:30.099290 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 22:32:30.099550 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":151,"threads_started":1}
I20260808 22:32:30.101280 14492 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e30354d85da48b398f1b740e6b6e01b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e30354d85da48b398f1b740e6b6e01b" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 39615 } }
I20260808 22:32:30.101598 14492 leader_election.cc:302] T 00000000000000000000000000000000 P 6e30354d85da48b398f1b740e6b6e01b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e30354d85da48b398f1b740e6b6e01b; no voters:
I20260808 22:32:30.101868 14492 leader_election.cc:288] T 00000000000000000000000000000000 P 6e30354d85da48b398f1b740e6b6e01b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:32:30.101954 14500 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e30354d85da48b398f1b740e6b6e01b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:32:30.102154 14500 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e30354d85da48b398f1b740e6b6e01b [term 1 LEADER]: Becoming Leader. State: Replica: 6e30354d85da48b398f1b740e6b6e01b, State: Running, Role: LEADER
I20260808 22:32:30.102507 14500 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e30354d85da48b398f1b740e6b6e01b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e30354d85da48b398f1b740e6b6e01b" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 39615 } }
I20260808 22:32:30.103003 14492 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e30354d85da48b398f1b740e6b6e01b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:32:30.104041 14501 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e30354d85da48b398f1b740e6b6e01b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e30354d85da48b398f1b740e6b6e01b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e30354d85da48b398f1b740e6b6e01b" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 39615 } } }
I20260808 22:32:30.104163 14501 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e30354d85da48b398f1b740e6b6e01b [sys.catalog]: This master's current role is: LEADER
I20260808 22:32:30.104552 14514 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:32:30.104501 14503 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e30354d85da48b398f1b740e6b6e01b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e30354d85da48b398f1b740e6b6e01b. Latest consensus state: current_term: 1 leader_uuid: "6e30354d85da48b398f1b740e6b6e01b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e30354d85da48b398f1b740e6b6e01b" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 39615 } } }
I20260808 22:32:30.105257 14503 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e30354d85da48b398f1b740e6b6e01b [sys.catalog]: This master's current role is: LEADER
I20260808 22:32:30.107534 14514 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:32:30.112795 14514 catalog_manager.cc:1347] Generated new cluster ID: 939f6461938b4b6ab86d4822fd37bd6b
I20260808 22:32:30.112847 14514 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:32:30.137862 14514 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:32:30.139029 14514 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:32:30.148113 14514 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e30354d85da48b398f1b740e6b6e01b: Generated new TSK 0
I20260808 22:32:30.148839 14514 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:32:30.151376 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.129:0
--local_ip_for_outbound_sockets=127.31.87.129
--webserver_interface=127.31.87.129
--webserver_port=0
--tserver_master_addrs=127.31.87.190:39615
--builtin_ntp_servers=127.31.87.148:43401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 22:32:30.275704 14529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:32:30.275933 14529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:32:30.275979 14529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:32:30.279973 14529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:32:30.280021 14529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:32:30.280099 14529 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.129
I20260808 22:32:30.284600 14529 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:43401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.87.129
--webserver_port=0
--tserver_master_addrs=127.31.87.190:39615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14529
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.129
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:32:30.285573 14529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:32:30.286832 14529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:32:30.295949 14537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:30.295982 14543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:30.295953 14536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:30.296142 14529 server_base.cc:1034] running on GCE node
I20260808 22:32:30.296659 14529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:32:30.297255 14529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:32:30.298404 14529 hybrid_clock.cc:648] HybridClock initialized: now 1786228350298380 us; error 40 us; skew 500 ppm
I20260808 22:32:30.300066 14529 webserver.cc:460] Webserver started at http://127.31.87.129:46195/ using document root <none> and password file <none>
I20260808 22:32:30.300586 14529 fs_manager.cc:359] Metadata directory not provided
I20260808 22:32:30.300650 14529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:32:30.300840 14529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:32:30.302749 14529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-0/data/instance:
uuid: "a8be78eb35364298afc7eed7840c75da"
format_stamp: "Formatted at 2026-08-08 22:32:30 on dist-test-slave-266d"
server_key: "275b81d235d6b2960ff1c377703842e6"
server_key_iv: "b3ff6b7a8da9541b779c5356a80fbe99"
server_key_version: "encryptionkey@0"
I20260808 22:32:30.303279 14529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-0/wal/instance:
uuid: "a8be78eb35364298afc7eed7840c75da"
format_stamp: "Formatted at 2026-08-08 22:32:30 on dist-test-slave-266d"
server_key: "275b81d235d6b2960ff1c377703842e6"
server_key_iv: "b3ff6b7a8da9541b779c5356a80fbe99"
server_key_version: "encryptionkey@0"
I20260808 22:32:30.306679 14529 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 22:32:30.308921 14549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:30.310022 14529 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:32:30.310129 14529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-0/wal
uuid: "a8be78eb35364298afc7eed7840c75da"
format_stamp: "Formatted at 2026-08-08 22:32:30 on dist-test-slave-266d"
server_key: "275b81d235d6b2960ff1c377703842e6"
server_key_iv: "b3ff6b7a8da9541b779c5356a80fbe99"
server_key_version: "encryptionkey@0"
I20260808 22:32:30.310216 14529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:32:30.325229 14529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:32:30.325805 14529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:32:30.325958 14529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:32:30.326504 14529 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:32:30.327489 14529 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:32:30.327538 14529 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:30.327579 14529 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:32:30.327600 14529 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:30.335469 14529 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.129:38305
I20260808 22:32:30.335530 14718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.129:38305 every 8 connection(s)
I20260808 22:32:30.336299 14529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
I20260808 22:32:30.338750 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 14529
I20260808 22:32:30.338871 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-0/wal/instance
I20260808 22:32:30.339198 32094 external_mini_cluster.cc:1428] Setting key 0d71abf81ffc98bc25dbe95d5a1268cc
I20260808 22:32:30.341253 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.130:0
--local_ip_for_outbound_sockets=127.31.87.130
--webserver_interface=127.31.87.130
--webserver_port=0
--tserver_master_addrs=127.31.87.190:39615
--builtin_ntp_servers=127.31.87.148:43401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 22:32:30.345820 14496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:30.337961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:40975 (local address 127.31.87.190:39615)
0808 22:32:30.338076 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:30.338080 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:30.339197 (+ 1117us) server_negotiation.cc:407] Connection header received
0808 22:32:30.340746 (+ 1549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:30.340749 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:30.340801 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:30.340867 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:30.341982 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:30.342499 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:30.343742 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:30.344080 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:30.344185 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:30.345336 (+ 1151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:30.345340 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:30.345343 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:32:30.345360 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:30.345384 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:32:30.345395 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:30.345463 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:30.345584 (+ 121us) server_negotiation.cc:299] Negotiation successful
0808 22:32:30.345652 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 22:32:30.346570 14721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:30.338292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:39615 (local address 127.31.87.129:40975)
0808 22:32:30.339015 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:30.339070 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:30.340528 (+ 1458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:30.341066 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:30.341075 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:30.341218 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:30.341842 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:30.341860 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:30.342621 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:30.342625 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:30.343625 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:30.343636 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:30.344259 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:30.344263 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:30.344413 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:30.345046 (+ 633us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:32:30.345048 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:30.345116 (+ 68us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:32:30.345187 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:32:30.345472 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:32:30.345478 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:32:30.345481 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 22:32:30.345574 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 22:32:30.345822 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":99,"threads_started":1}
I20260808 22:32:30.347678 14719 heartbeater.cc:347] Connected to a master server at 127.31.87.190:39615
I20260808 22:32:30.347930 14719 heartbeater.cc:464] Registering TS with master...
I20260808 22:32:30.348517 14719 heartbeater.cc:511] Master 127.31.87.190:39615 requested a full tablet report, sending...
I20260808 22:32:30.350154 14440 ts_manager.cc:194] Registered new tserver with Master: a8be78eb35364298afc7eed7840c75da (127.31.87.129:38305)
I20260808 22:32:30.351346 14440 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.87.129:40975
W20260808 22:32:30.452171 14722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:32:30.452425 14722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:32:30.452471 14722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:32:30.456391 14722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:32:30.456436 14722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:32:30.456491 14722 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.130
I20260808 22:32:30.461017 14722 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:43401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.87.130
--webserver_port=0
--tserver_master_addrs=127.31.87.190:39615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14722
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.130
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:32:30.461953 14722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:32:30.463220 14722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:32:30.472278 14732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:30.472270 14737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:30.472384 14722 server_base.cc:1034] running on GCE node
W20260808 22:32:30.472265 14733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:30.472836 14722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:32:30.473444 14722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:32:30.474593 14722 hybrid_clock.cc:648] HybridClock initialized: now 1786228350474576 us; error 26 us; skew 500 ppm
I20260808 22:32:30.476243 14722 webserver.cc:460] Webserver started at http://127.31.87.130:43437/ using document root <none> and password file <none>
I20260808 22:32:30.476765 14722 fs_manager.cc:359] Metadata directory not provided
I20260808 22:32:30.476823 14722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:32:30.476984 14722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:32:30.478961 14722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-1/data/instance:
uuid: "e47934054981450c935b74838c7ef8bd"
format_stamp: "Formatted at 2026-08-08 22:32:30 on dist-test-slave-266d"
server_key: "6d47bd2a24a3d31c01575c79c035d0f0"
server_key_iv: "b072be301bb5577b1dd7c733d88822c1"
server_key_version: "encryptionkey@0"
I20260808 22:32:30.479462 14722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-1/wal/instance:
uuid: "e47934054981450c935b74838c7ef8bd"
format_stamp: "Formatted at 2026-08-08 22:32:30 on dist-test-slave-266d"
server_key: "6d47bd2a24a3d31c01575c79c035d0f0"
server_key_iv: "b072be301bb5577b1dd7c733d88822c1"
server_key_version: "encryptionkey@0"
I20260808 22:32:30.482834 14722 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 22:32:30.485074 14746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:30.486233 14722 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 22:32:30.486335 14722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-1/wal
uuid: "e47934054981450c935b74838c7ef8bd"
format_stamp: "Formatted at 2026-08-08 22:32:30 on dist-test-slave-266d"
server_key: "6d47bd2a24a3d31c01575c79c035d0f0"
server_key_iv: "b072be301bb5577b1dd7c733d88822c1"
server_key_version: "encryptionkey@0"
I20260808 22:32:30.486425 14722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:32:30.503101 14722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:32:30.503738 14722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:32:30.503892 14722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:32:30.504405 14722 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:32:30.505331 14722 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:32:30.505380 14722 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:30.505431 14722 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:32:30.505460 14722 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:30.513506 14722 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.130:40181
I20260808 22:32:30.513532 14925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.130:40181 every 8 connection(s)
I20260808 22:32:30.514379 14722 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
I20260808 22:32:30.517839 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 14722
I20260808 22:32:30.517948 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-1/wal/instance
I20260808 22:32:30.518216 32094 external_mini_cluster.cc:1428] Setting key 476d97000e89f9362b7d7653ea1ffada
I20260808 22:32:30.520136 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.131:0
--local_ip_for_outbound_sockets=127.31.87.131
--webserver_interface=127.31.87.131
--webserver_port=0
--tserver_master_addrs=127.31.87.190:39615
--builtin_ntp_servers=127.31.87.148:43401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 22:32:30.523207 14496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:30.516037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:45281 (local address 127.31.87.190:39615)
0808 22:32:30.516172 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:30.516176 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:30.517185 (+ 1009us) server_negotiation.cc:407] Connection header received
0808 22:32:30.518670 (+ 1485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:30.518673 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:30.518723 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:30.518806 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:30.519663 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:30.520186 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:30.521415 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:30.521666 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:30.521774 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:30.522740 (+ 966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:30.522744 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:30.522746 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:32:30.522760 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:30.522783 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:32:30.522792 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:30.522860 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:30.523004 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 22:32:30.523065 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 22:32:30.524086 14929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:30.516404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:39615 (local address 127.31.87.130:45281)
0808 22:32:30.517015 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:30.517053 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:30.518453 (+ 1400us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:30.518988 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:30.518995 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:30.519077 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:30.519537 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:30.519550 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:30.520381 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:30.520384 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:30.521302 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:30.521312 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:30.521829 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:30.521832 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:30.521975 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:30.522501 (+ 526us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:32:30.522503 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:30.522541 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:32:30.522601 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:32:30.522873 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:32:30.522878 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:32:30.522882 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:32:30.522981 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 22:32:30.523244 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":105,"threads_started":1}
I20260808 22:32:30.525252 14926 heartbeater.cc:347] Connected to a master server at 127.31.87.190:39615
I20260808 22:32:30.525475 14926 heartbeater.cc:464] Registering TS with master...
I20260808 22:32:30.525936 14926 heartbeater.cc:511] Master 127.31.87.190:39615 requested a full tablet report, sending...
I20260808 22:32:30.526890 14440 ts_manager.cc:194] Registered new tserver with Master: e47934054981450c935b74838c7ef8bd (127.31.87.130:40181)
I20260808 22:32:30.527412 14440 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.87.130:45281
W20260808 22:32:30.629793 14931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:32:30.630026 14931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:32:30.630074 14931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:32:30.634032 14931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:32:30.634078 14931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:32:30.634146 14931 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.131
I20260808 22:32:30.638674 14931 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:43401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.87.131
--webserver_port=0
--tserver_master_addrs=127.31.87.190:39615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14931
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.131
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:32:30.639622 14931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:32:30.640864 14931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:32:30.649891 14937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:30.649905 14938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:30.649905 14942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:30.650319 14931 server_base.cc:1034] running on GCE node
I20260808 22:32:30.650664 14931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:32:30.651242 14931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:32:30.652390 14931 hybrid_clock.cc:648] HybridClock initialized: now 1786228350652363 us; error 36 us; skew 500 ppm
I20260808 22:32:30.654098 14931 webserver.cc:460] Webserver started at http://127.31.87.131:44443/ using document root <none> and password file <none>
I20260808 22:32:30.654624 14931 fs_manager.cc:359] Metadata directory not provided
I20260808 22:32:30.654685 14931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:32:30.654901 14931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:32:30.656816 14931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-2/data/instance:
uuid: "3e31bd48c62d4d06b929a92ecb688625"
format_stamp: "Formatted at 2026-08-08 22:32:30 on dist-test-slave-266d"
server_key: "44f8083802b70777cb2107b8a389d116"
server_key_iv: "663ceebd06425dd605765419e37c8646"
server_key_version: "encryptionkey@0"
I20260808 22:32:30.657353 14931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-2/wal/instance:
uuid: "3e31bd48c62d4d06b929a92ecb688625"
format_stamp: "Formatted at 2026-08-08 22:32:30 on dist-test-slave-266d"
server_key: "44f8083802b70777cb2107b8a389d116"
server_key_iv: "663ceebd06425dd605765419e37c8646"
server_key_version: "encryptionkey@0"
I20260808 22:32:30.660677 14931 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 22:32:30.663013 14953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:30.664132 14931 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:32:30.664232 14931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-2/wal
uuid: "3e31bd48c62d4d06b929a92ecb688625"
format_stamp: "Formatted at 2026-08-08 22:32:30 on dist-test-slave-266d"
server_key: "44f8083802b70777cb2107b8a389d116"
server_key_iv: "663ceebd06425dd605765419e37c8646"
server_key_version: "encryptionkey@0"
I20260808 22:32:30.664327 14931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:32:30.684876 14931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:32:30.685492 14931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:32:30.685660 14931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:32:30.686173 14931 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:32:30.687036 14931 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:32:30.687081 14931 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:30.687120 14931 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:32:30.687150 14931 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:30.695259 14931 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.131:36753
I20260808 22:32:30.695283 15134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.131:36753 every 8 connection(s)
I20260808 22:32:30.696077 14931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
I20260808 22:32:30.696734 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 14931
I20260808 22:32:30.696847 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-2/wal/instance
I20260808 22:32:30.697165 32094 external_mini_cluster.cc:1428] Setting key 6ed22212289d2d5de10b2d9289a3fb3c
I20260808 22:32:30.704947 14496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:30.698007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:48911 (local address 127.31.87.190:39615)
0808 22:32:30.698120 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:30.698124 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:30.699376 (+ 1252us) server_negotiation.cc:407] Connection header received
0808 22:32:30.700710 (+ 1334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:30.700713 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:30.700759 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:30.700830 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:30.701719 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:30.702246 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:30.703342 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:30.703566 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:30.703651 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:30.704594 (+ 943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:30.704598 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:30.704600 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:32:30.704613 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:30.704631 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:32:30.704639 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:30.704691 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:30.704792 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 22:32:30.704839 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 22:32:30.705763 15137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:30.698497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:39615 (local address 127.31.87.131:48911)
0808 22:32:30.699217 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:30.699278 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:30.700504 (+ 1226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:30.700964 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:30.700970 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:30.701048 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:30.701573 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:30.701587 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:30.702333 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:30.702335 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:30.703220 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:30.703230 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:30.703675 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:30.703678 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:30.703807 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:30.704381 (+ 574us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:32:30.704383 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:30.704419 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:32:30.704479 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:32:30.704698 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:32:30.704702 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:32:30.704705 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 22:32:30.704787 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 22:32:30.704987 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":102,"threads_started":1}
I20260808 22:32:30.706728 15135 heartbeater.cc:347] Connected to a master server at 127.31.87.190:39615
I20260808 22:32:30.706918 15135 heartbeater.cc:464] Registering TS with master...
I20260808 22:32:30.707352 15135 heartbeater.cc:511] Master 127.31.87.190:39615 requested a full tablet report, sending...
I20260808 22:32:30.708243 14440 ts_manager.cc:194] Registered new tserver with Master: 3e31bd48c62d4d06b929a92ecb688625 (127.31.87.131:36753)
I20260808 22:32:30.708752 14440 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.87.131:48911
I20260808 22:32:30.710932 32094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 22:32:30.713644 14496 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:30.712684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51696 (local address 127.31.87.190:39615)
0808 22:32:30.712815 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:30.712820 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:30.712944 (+ 124us) server_negotiation.cc:407] Connection header received
0808 22:32:30.713148 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:30.713151 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:30.713195 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:30.713258 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:30.713529 (+ 271us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51696: BlockingRecv error: recv got EOF from 127.0.0.1:51696 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 22:32:30.713763 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 14529
I20260808 22:32:30.727689 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 14722
I20260808 22:32:30.745225 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 14931
I20260808 22:32:30.756238 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 14389
2026-08-08T22:32:30Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (887 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-08T22:32:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:32:30Z Disabled control of system clock
I20260808 22:32:30.793182 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:45439
--webserver_interface=127.31.87.190
--webserver_port=0
--builtin_ntp_servers=127.31.87.148:43657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:45439
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 22:32:30.899477 15150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:32:30.899725 15150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:32:30.899760 15150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:32:30.903621 15150 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:32:30.903667 15150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:32:30.903680 15150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:32:30.903695 15150 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:32:30.903708 15150 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:32:30.908255 15150 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:43657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:45439
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:45439
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.87.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15150
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:32:30.909210 15150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:32:30.910444 15150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:32:30.918349 15164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:30.918390 15162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:30.918370 15166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:30.918547 15150 server_base.cc:1034] running on GCE node
I20260808 22:32:30.919183 15150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:32:30.920190 15150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:32:30.921344 15150 hybrid_clock.cc:648] HybridClock initialized: now 1786228350921326 us; error 34 us; skew 500 ppm
I20260808 22:32:30.923012 15150 webserver.cc:460] Webserver started at http://127.31.87.190:44559/ using document root <none> and password file <none>
I20260808 22:32:30.923534 15150 fs_manager.cc:359] Metadata directory not provided
I20260808 22:32:30.923609 15150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:32:30.923811 15150 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:32:30.925772 15150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/master-0/data/instance:
uuid: "31f2a55d848e4e1382914e7b674a8940"
format_stamp: "Formatted at 2026-08-08 22:32:30 on dist-test-slave-266d"
server_key: "caea640b51e34ba85a3e3cb3b2827ab3"
server_key_iv: "31514cbfd4896496aa75c3cb10cfe1ea"
server_key_version: "encryptionkey@0"
I20260808 22:32:30.926290 15150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/master-0/wal/instance:
uuid: "31f2a55d848e4e1382914e7b674a8940"
format_stamp: "Formatted at 2026-08-08 22:32:30 on dist-test-slave-266d"
server_key: "caea640b51e34ba85a3e3cb3b2827ab3"
server_key_iv: "31514cbfd4896496aa75c3cb10cfe1ea"
server_key_version: "encryptionkey@0"
I20260808 22:32:30.929754 15150 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:32:30.931880 15176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:30.932972 15150 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:32:30.933081 15150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "31f2a55d848e4e1382914e7b674a8940"
format_stamp: "Formatted at 2026-08-08 22:32:30 on dist-test-slave-266d"
server_key: "caea640b51e34ba85a3e3cb3b2827ab3"
server_key_iv: "31514cbfd4896496aa75c3cb10cfe1ea"
server_key_version: "encryptionkey@0"
I20260808 22:32:30.933207 15150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:32:30.937388 15150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:32:30.937927 15150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:32:30.938076 15150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:32:30.944139 15150 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.190:45439
I20260808 22:32:30.944141 15259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.190:45439 every 8 connection(s)
I20260808 22:32:30.944949 15150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:32:30.948112 15260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:32:30.948135 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 15150
I20260808 22:32:30.948239 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/master-0/wal/instance
I20260808 22:32:30.948534 32094 external_mini_cluster.cc:1428] Setting key e0c04e217bc961827014169998a85099
I20260808 22:32:30.954106 15266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:30.949847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59314 (local address 127.31.87.190:45439)
0808 22:32:30.950509 (+ 662us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:30.950527 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:30.950568 (+ 41us) server_negotiation.cc:407] Connection header received
0808 22:32:30.951291 (+ 723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:30.951315 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:30.951728 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:30.952079 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:30.952725 (+ 646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:30.952731 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:30.952745 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:32:30.952755 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:30.952814 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:32:30.952836 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:30.952970 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:30.953034 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 22:32:30.953297 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":104,"threads_started":1}
I20260808 22:32:30.954926 15260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31f2a55d848e4e1382914e7b674a8940: Bootstrap starting.
I20260808 22:32:30.957137 15260 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 31f2a55d848e4e1382914e7b674a8940: Neither blocks nor log segments found. Creating new log.
I20260808 22:32:30.957805 15260 log.cc:824] T 00000000000000000000000000000000 P 31f2a55d848e4e1382914e7b674a8940: Log is configured to *not* fsync() on all Append() calls
I20260808 22:32:30.959479 15260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31f2a55d848e4e1382914e7b674a8940: No bootstrap required, opened a new log
I20260808 22:32:30.962841 15260 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31f2a55d848e4e1382914e7b674a8940 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31f2a55d848e4e1382914e7b674a8940" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45439 } }
I20260808 22:32:30.963037 15260 raft_consensus.cc:374] T 00000000000000000000000000000000 P 31f2a55d848e4e1382914e7b674a8940 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:32:30.963083 15260 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31f2a55d848e4e1382914e7b674a8940 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31f2a55d848e4e1382914e7b674a8940, State: Initialized, Role: FOLLOWER
I20260808 22:32:30.963414 15260 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31f2a55d848e4e1382914e7b674a8940 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31f2a55d848e4e1382914e7b674a8940" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45439 } }
I20260808 22:32:30.963516 15260 raft_consensus.cc:388] T 00000000000000000000000000000000 P 31f2a55d848e4e1382914e7b674a8940 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:32:30.963558 15260 raft_consensus.cc:482] T 00000000000000000000000000000000 P 31f2a55d848e4e1382914e7b674a8940 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:32:30.963624 15260 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31f2a55d848e4e1382914e7b674a8940 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:32:30.964392 15260 raft_consensus.cc:504] T 00000000000000000000000000000000 P 31f2a55d848e4e1382914e7b674a8940 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31f2a55d848e4e1382914e7b674a8940" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45439 } }
I20260808 22:32:30.964672 15260 leader_election.cc:302] T 00000000000000000000000000000000 P 31f2a55d848e4e1382914e7b674a8940 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 31f2a55d848e4e1382914e7b674a8940; no voters:
I20260808 22:32:30.964913 15260 leader_election.cc:288] T 00000000000000000000000000000000 P 31f2a55d848e4e1382914e7b674a8940 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:32:30.965009 15268 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 31f2a55d848e4e1382914e7b674a8940 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:32:30.965245 15268 raft_consensus.cc:686] T 00000000000000000000000000000000 P 31f2a55d848e4e1382914e7b674a8940 [term 1 LEADER]: Becoming Leader. State: Replica: 31f2a55d848e4e1382914e7b674a8940, State: Running, Role: LEADER
I20260808 22:32:30.965538 15268 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31f2a55d848e4e1382914e7b674a8940 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31f2a55d848e4e1382914e7b674a8940" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45439 } }
I20260808 22:32:30.966166 15260 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31f2a55d848e4e1382914e7b674a8940 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:32:30.967183 15276 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31f2a55d848e4e1382914e7b674a8940 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "31f2a55d848e4e1382914e7b674a8940" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31f2a55d848e4e1382914e7b674a8940" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45439 } } }
I20260808 22:32:30.967203 15278 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31f2a55d848e4e1382914e7b674a8940 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31f2a55d848e4e1382914e7b674a8940. Latest consensus state: current_term: 1 leader_uuid: "31f2a55d848e4e1382914e7b674a8940" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31f2a55d848e4e1382914e7b674a8940" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45439 } } }
I20260808 22:32:30.967301 15276 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31f2a55d848e4e1382914e7b674a8940 [sys.catalog]: This master's current role is: LEADER
I20260808 22:32:30.967310 15278 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31f2a55d848e4e1382914e7b674a8940 [sys.catalog]: This master's current role is: LEADER
I20260808 22:32:30.968290 15289 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:32:30.970834 15289 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:32:30.976022 15289 catalog_manager.cc:1347] Generated new cluster ID: 4f5b5e00a4aa4a65b3069f9d67670d3d
I20260808 22:32:30.976082 15289 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:32:31.000510 15289 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:32:31.000813 15289 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:32:31.009639 15289 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 31f2a55d848e4e1382914e7b674a8940: Generated new TSK 0
I20260808 22:32:31.010267 15289 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:32:31.015846 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.129:0
--local_ip_for_outbound_sockets=127.31.87.129
--webserver_interface=127.31.87.129
--webserver_port=0
--tserver_master_addrs=127.31.87.190:45439
--builtin_ntp_servers=127.31.87.148:43657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 22:32:31.148079 15306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:32:31.148317 15306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:32:31.148355 15306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:32:31.152230 15306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:32:31.152275 15306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:32:31.152326 15306 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.129
I20260808 22:32:31.156801 15306 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:43657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.87.129
--webserver_port=0
--tserver_master_addrs=127.31.87.190:45439
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15306
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.129
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:32:31.157724 15306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:32:31.158967 15306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:32:31.167912 15316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:31.167929 15312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:31.167913 15320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:31.168290 15306 server_base.cc:1034] running on GCE node
I20260808 22:32:31.168635 15306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:32:31.169204 15306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:32:31.170352 15306 hybrid_clock.cc:648] HybridClock initialized: now 1786228351170339 us; error 26 us; skew 500 ppm
I20260808 22:32:31.172011 15306 webserver.cc:460] Webserver started at http://127.31.87.129:37851/ using document root <none> and password file <none>
I20260808 22:32:31.172513 15306 fs_manager.cc:359] Metadata directory not provided
I20260808 22:32:31.172571 15306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:32:31.172734 15306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:32:31.174620 15306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-0/data/instance:
uuid: "f18ed05857cc4b8f852d93627ae54a06"
format_stamp: "Formatted at 2026-08-08 22:32:31 on dist-test-slave-266d"
server_key: "3ffe3948bdc6756fa37edf7f57f905ac"
server_key_iv: "70dd77d61bb61815da9e37c2dd285b4a"
server_key_version: "encryptionkey@0"
I20260808 22:32:31.175122 15306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-0/wal/instance:
uuid: "f18ed05857cc4b8f852d93627ae54a06"
format_stamp: "Formatted at 2026-08-08 22:32:31 on dist-test-slave-266d"
server_key: "3ffe3948bdc6756fa37edf7f57f905ac"
server_key_iv: "70dd77d61bb61815da9e37c2dd285b4a"
server_key_version: "encryptionkey@0"
I20260808 22:32:31.178490 15306 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 22:32:31.180696 15330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:31.181797 15306 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:32:31.181901 15306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-0/wal
uuid: "f18ed05857cc4b8f852d93627ae54a06"
format_stamp: "Formatted at 2026-08-08 22:32:31 on dist-test-slave-266d"
server_key: "3ffe3948bdc6756fa37edf7f57f905ac"
server_key_iv: "70dd77d61bb61815da9e37c2dd285b4a"
server_key_version: "encryptionkey@0"
I20260808 22:32:31.181993 15306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:32:31.186196 15306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:32:31.186743 15306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:32:31.186900 15306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:32:31.187395 15306 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:32:31.188225 15306 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:32:31.188272 15306 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:31.188321 15306 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:32:31.188347 15306 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:31.196393 15306 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.129:34503
I20260808 22:32:31.196415 15497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.129:34503 every 8 connection(s)
I20260808 22:32:31.197345 15306 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
I20260808 22:32:31.202903 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 15306
I20260808 22:32:31.203024 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-0/wal/instance
I20260808 22:32:31.203153 15266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:31.199069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:37003 (local address 127.31.87.190:45439)
0808 22:32:31.199181 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:31.199185 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:31.200195 (+ 1010us) server_negotiation.cc:407] Connection header received
0808 22:32:31.201517 (+ 1322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:31.201520 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:31.201570 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:31.201642 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:31.202656 (+ 1014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:31.202660 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:31.202663 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:32:31.202688 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:31.202711 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:32:31.202722 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:31.202807 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:31.202976 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 22:32:31.203023 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 22:32:31.203358 32094 external_mini_cluster.cc:1428] Setting key 15d4136297ec5f458954f5557dd32f86
I20260808 22:32:31.204077 15504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:31.199429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45439 (local address 127.31.87.129:37003)
0808 22:32:31.200059 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:31.200097 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:31.201310 (+ 1213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:31.201813 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:31.201820 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:31.201903 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:31.202415 (+ 512us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:32:31.202416 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:31.202454 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:32:31.202517 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:32:31.202820 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:32:31.202829 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:32:31.202835 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 22:32:31.203005 (+ 170us) client_negotiation.cc:240] Negotiation successful
0808 22:32:31.203255 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":121,"threads_started":1}
I20260808 22:32:31.205206 15499 heartbeater.cc:347] Connected to a master server at 127.31.87.190:45439
I20260808 22:32:31.205366 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.130:0
--local_ip_for_outbound_sockets=127.31.87.130
--webserver_interface=127.31.87.130
--webserver_port=0
--tserver_master_addrs=127.31.87.190:45439
--builtin_ntp_servers=127.31.87.148:43657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 22:32:31.205441 15499 heartbeater.cc:464] Registering TS with master...
I20260808 22:32:31.205835 15499 heartbeater.cc:511] Master 127.31.87.190:45439 requested a full tablet report, sending...
I20260808 22:32:31.207342 15204 ts_manager.cc:194] Registered new tserver with Master: f18ed05857cc4b8f852d93627ae54a06 (127.31.87.129:34503)
W20260808 22:32:31.315119 15505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:32:31.315368 15505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:32:31.315421 15505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:32:31.319618 15505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:32:31.319667 15505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:32:31.319741 15505 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.130
I20260808 22:32:31.324348 15505 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:43657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.87.130
--webserver_port=0
--tserver_master_addrs=127.31.87.190:45439
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15505
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.130
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:32:31.325342 15505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:32:31.326596 15505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:32:31.335613 15511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:31.335683 15505 server_base.cc:1034] running on GCE node
W20260808 22:32:31.335590 15514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:31.335623 15510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:31.336196 15505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:32:31.336792 15505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:32:31.337945 15505 hybrid_clock.cc:648] HybridClock initialized: now 1786228351337932 us; error 24 us; skew 500 ppm
I20260808 22:32:31.339530 15505 webserver.cc:460] Webserver started at http://127.31.87.130:39107/ using document root <none> and password file <none>
I20260808 22:32:31.340041 15505 fs_manager.cc:359] Metadata directory not provided
I20260808 22:32:31.340104 15505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:32:31.340292 15505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:32:31.342235 15505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-1/data/instance:
uuid: "748757cc5f7e4d469f38e8fc22fcc261"
format_stamp: "Formatted at 2026-08-08 22:32:31 on dist-test-slave-266d"
server_key: "d169a4d7f746a2872bcfd6e6dc9277f7"
server_key_iv: "3eb9818d83ea944cbc4dc9916fb87454"
server_key_version: "encryptionkey@0"
I20260808 22:32:31.342736 15505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-1/wal/instance:
uuid: "748757cc5f7e4d469f38e8fc22fcc261"
format_stamp: "Formatted at 2026-08-08 22:32:31 on dist-test-slave-266d"
server_key: "d169a4d7f746a2872bcfd6e6dc9277f7"
server_key_iv: "3eb9818d83ea944cbc4dc9916fb87454"
server_key_version: "encryptionkey@0"
I20260808 22:32:31.346176 15505 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:32:31.348510 15526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:31.349644 15505 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:32:31.349747 15505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-1/wal
uuid: "748757cc5f7e4d469f38e8fc22fcc261"
format_stamp: "Formatted at 2026-08-08 22:32:31 on dist-test-slave-266d"
server_key: "d169a4d7f746a2872bcfd6e6dc9277f7"
server_key_iv: "3eb9818d83ea944cbc4dc9916fb87454"
server_key_version: "encryptionkey@0"
I20260808 22:32:31.349840 15505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:32:31.354194 15505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:32:31.354822 15505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:32:31.354974 15505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:32:31.355469 15505 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:32:31.356321 15505 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:32:31.356367 15505 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:31.356415 15505 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:32:31.356443 15505 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:31.364691 15505 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.130:32957
I20260808 22:32:31.364710 15705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.130:32957 every 8 connection(s)
I20260808 22:32:31.365566 15505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
I20260808 22:32:31.370930 15266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:31.367242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:59727 (local address 127.31.87.190:45439)
0808 22:32:31.367357 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:31.367360 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:31.368387 (+ 1027us) server_negotiation.cc:407] Connection header received
0808 22:32:31.369615 (+ 1228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:31.369618 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:31.369657 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:31.369725 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:31.370604 (+ 879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:31.370608 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:31.370610 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:32:31.370622 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:31.370641 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:32:31.370650 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:31.370719 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:31.370791 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 22:32:31.370825 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 22:32:31.371753 15708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:31.367645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45439 (local address 127.31.87.130:59727)
0808 22:32:31.368254 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:31.368291 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:31.369445 (+ 1154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:31.369845 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:31.369851 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:31.369932 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:31.370375 (+ 443us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:32:31.370377 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:31.370412 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:32:31.370474 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:32:31.370703 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:32:31.370707 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:32:31.370710 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 22:32:31.370819 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 22:32:31.370998 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":101,"threads_started":1}
I20260808 22:32:31.371832 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 15505
I20260808 22:32:31.371945 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-1/wal/instance
I20260808 22:32:31.372226 32094 external_mini_cluster.cc:1428] Setting key fb438efddd6c88ad01e5fcccf6b85ddd
I20260808 22:32:31.372802 15706 heartbeater.cc:347] Connected to a master server at 127.31.87.190:45439
I20260808 22:32:31.373000 15706 heartbeater.cc:464] Registering TS with master...
I20260808 22:32:31.373379 15706 heartbeater.cc:511] Master 127.31.87.190:45439 requested a full tablet report, sending...
I20260808 22:32:31.374050 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.131:0
--local_ip_for_outbound_sockets=127.31.87.131
--webserver_interface=127.31.87.131
--webserver_port=0
--tserver_master_addrs=127.31.87.190:45439
--builtin_ntp_servers=127.31.87.148:43657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 22:32:31.374346 15204 ts_manager.cc:194] Registered new tserver with Master: 748757cc5f7e4d469f38e8fc22fcc261 (127.31.87.130:32957)
W20260808 22:32:31.482988 15709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:32:31.483206 15709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:32:31.483240 15709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:32:31.487176 15709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:32:31.487223 15709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:32:31.487277 15709 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.131
I20260808 22:32:31.491770 15709 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:43657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.87.131
--webserver_port=0
--tserver_master_addrs=127.31.87.190:45439
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15709
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.131
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:32:31.492641 15709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:32:31.493866 15709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:32:31.502736 15717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:31.502802 15709 server_base.cc:1034] running on GCE node
W20260808 22:32:31.502743 15720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:31.502743 15718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:31.503340 15709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:32:31.503916 15709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:32:31.505061 15709 hybrid_clock.cc:648] HybridClock initialized: now 1786228351505044 us; error 28 us; skew 500 ppm
I20260808 22:32:31.506680 15709 webserver.cc:460] Webserver started at http://127.31.87.131:43229/ using document root <none> and password file <none>
I20260808 22:32:31.507207 15709 fs_manager.cc:359] Metadata directory not provided
I20260808 22:32:31.507269 15709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:32:31.507460 15709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:32:31.509375 15709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-2/data/instance:
uuid: "efab3e45cb464086aa55072d05d03442"
format_stamp: "Formatted at 2026-08-08 22:32:31 on dist-test-slave-266d"
server_key: "3a527b54f9029f8f69a0aa04c2c8da07"
server_key_iv: "c6ac2958ec16b0bcd72e806e565dfacb"
server_key_version: "encryptionkey@0"
I20260808 22:32:31.509910 15709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-2/wal/instance:
uuid: "efab3e45cb464086aa55072d05d03442"
format_stamp: "Formatted at 2026-08-08 22:32:31 on dist-test-slave-266d"
server_key: "3a527b54f9029f8f69a0aa04c2c8da07"
server_key_iv: "c6ac2958ec16b0bcd72e806e565dfacb"
server_key_version: "encryptionkey@0"
I20260808 22:32:31.513247 15709 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 22:32:31.515481 15728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:31.516568 15709 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:32:31.516662 15709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-2/wal
uuid: "efab3e45cb464086aa55072d05d03442"
format_stamp: "Formatted at 2026-08-08 22:32:31 on dist-test-slave-266d"
server_key: "3a527b54f9029f8f69a0aa04c2c8da07"
server_key_iv: "c6ac2958ec16b0bcd72e806e565dfacb"
server_key_version: "encryptionkey@0"
I20260808 22:32:31.516764 15709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:32:31.521209 15709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:32:31.521790 15709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:32:31.521957 15709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:32:31.522473 15709 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:32:31.523351 15709 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:32:31.523402 15709 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:31.523458 15709 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:32:31.523491 15709 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:31.532341 15709 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.131:41007
I20260808 22:32:31.532351 15885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.131:41007 every 8 connection(s)
I20260808 22:32:31.533268 15709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
I20260808 22:32:31.538815 15266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:31.534893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:44317 (local address 127.31.87.190:45439)
0808 22:32:31.535051 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:31.535056 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:31.536023 (+ 967us) server_negotiation.cc:407] Connection header received
0808 22:32:31.537393 (+ 1370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:31.537396 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:31.537433 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:31.537504 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:31.538453 (+ 949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:31.538456 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:31.538458 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:32:31.538470 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:31.538488 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:32:31.538496 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:31.538559 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:31.538663 (+ 104us) server_negotiation.cc:299] Negotiation successful
0808 22:32:31.538697 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 22:32:31.539587 15893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:31.535237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45439 (local address 127.31.87.131:44317)
0808 22:32:31.535866 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:31.535906 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:31.537202 (+ 1296us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:31.537648 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:31.537655 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:31.537738 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:31.538209 (+ 471us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:32:31.538210 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:31.538247 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:32:31.538314 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:32:31.538557 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:32:31.538563 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:32:31.538569 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 22:32:31.538695 (+ 126us) client_negotiation.cc:240] Negotiation successful
0808 22:32:31.538876 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":105,"threads_started":1}
I20260808 22:32:31.540536 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 15709
I20260808 22:32:31.540658 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786228291013016-32094-0/minicluster-data/ts-2/wal/instance
I20260808 22:32:31.540688 15887 heartbeater.cc:347] Connected to a master server at 127.31.87.190:45439
I20260808 22:32:31.540910 15887 heartbeater.cc:464] Registering TS with master...
I20260808 22:32:31.540932 32094 external_mini_cluster.cc:1428] Setting key 1078517ed328b5a5438a802ee8e2f02d
I20260808 22:32:31.541347 15887 heartbeater.cc:511] Master 127.31.87.190:45439 requested a full tablet report, sending...
I20260808 22:32:31.542464 15204 ts_manager.cc:194] Registered new tserver with Master: efab3e45cb464086aa55072d05d03442 (127.31.87.131:41007)
I20260808 22:32:31.554489 32094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 22:32:31.557060 15266 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:32:31.556230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59322 (local address 127.31.87.190:45439)
0808 22:32:31.556355 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:31.556359 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:31.556515 (+ 156us) server_negotiation.cc:407] Connection header received
0808 22:32:31.556616 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:31.556619 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:31.556654 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:31.556717 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:31.556942 (+ 225us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59322: BlockingRecv error: recv got EOF from 127.0.0.1:59322 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 22:32:31.557212 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 15306
I20260808 22:32:31.572832 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 15505
I20260808 22:32:31.596408 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 15709
I20260808 22:32:31.606599 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 15150
2026-08-08T22:32:31Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (847 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:32:31 dist-test-slave-266d krb5kdc[15907](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:32:31 dist-test-slave-266d krb5kdc[15907](info): set up 2 sockets
Aug 08 22:32:31 dist-test-slave-266d krb5kdc[15907](info): commencing operation
krb5kdc: starting...
W20260808 22:32:32.764640 32094 mini_kdc.cc:121] Time spent starting KDC: real 1.132s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Aug 08 22:32:32 dist-test-slave-266d krb5kdc[15907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228352, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:32:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:32:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.87.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
I20260808 22:32:32.883955 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:46251
--webserver_interface=127.31.87.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:36035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:46251
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:32:32.991598 15923 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:32:32.991843 15923 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:32:32.991880 15923 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:32:32.995759 15923 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:32:32.995805 15923 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:32:32.995823 15923 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:32:32.995843 15923 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:32:32.995863 15923 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:32:33.000828 15923 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:36035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:46251
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:46251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.87.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15923
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:32:33.001745 15923 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:32:33.002646 15923 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:32:33.010669 15930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:33.010686 15933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:33.010694 15929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:33.010967 15923 server_base.cc:1034] running on GCE node
I20260808 22:32:33.011466 15923 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:32:33.012485 15923 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:32:33.013638 15923 hybrid_clock.cc:648] HybridClock initialized: now 1786228353013617 us; error 37 us; skew 500 ppm
Aug 08 22:32:33 dist-test-slave-266d krb5kdc[15907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228353, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:32:33.020244 15923 init.cc:377] Logged in from keytab as kudu/127.31.87.190@KRBTEST.COM (short username kudu)
I20260808 22:32:33.021212 15923 webserver.cc:460] Webserver started at http://127.31.87.190:33795/ using document root <none> and password file <none>
I20260808 22:32:33.021754 15923 fs_manager.cc:359] Metadata directory not provided
I20260808 22:32:33.021818 15923 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:32:33.022022 15923 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:32:33.024005 15923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/master-0/data/instance:
uuid: "b590ee15210b4877b85eca352f460558"
format_stamp: "Formatted at 2026-08-08 22:32:33 on dist-test-slave-266d"
server_key: "e17fde8e2127b3bb1252af4c307e5021"
server_key_iv: "aae4057be4344957bd55eefcbd2a4f2f"
server_key_version: "encryptionkey@0"
I20260808 22:32:33.024535 15923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/master-0/wal/instance:
uuid: "b590ee15210b4877b85eca352f460558"
format_stamp: "Formatted at 2026-08-08 22:32:33 on dist-test-slave-266d"
server_key: "e17fde8e2127b3bb1252af4c307e5021"
server_key_iv: "aae4057be4344957bd55eefcbd2a4f2f"
server_key_version: "encryptionkey@0"
I20260808 22:32:33.028064 15923 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260808 22:32:33.030385 15946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:33.031510 15923 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:32:33.031630 15923 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "b590ee15210b4877b85eca352f460558"
format_stamp: "Formatted at 2026-08-08 22:32:33 on dist-test-slave-266d"
server_key: "e17fde8e2127b3bb1252af4c307e5021"
server_key_iv: "aae4057be4344957bd55eefcbd2a4f2f"
server_key_version: "encryptionkey@0"
I20260808 22:32:33.031718 15923 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:32:33.043262 15923 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:32:33.046190 15923 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:32:33.046360 15923 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:32:33.053050 15923 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.190:46251
I20260808 22:32:33.053370 16030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.190:46251 every 8 connection(s)
I20260808 22:32:33.054028 15923 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:32:33.057498 16031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:32:33.059203 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 15923
I20260808 22:32:33.059321 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/master-0/wal/instance
I20260808 22:32:33.059612 32094 external_mini_cluster.cc:1428] Setting key cb55f4a40b0d9991387885661a547a0b
I20260808 22:32:33.063742 16031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b590ee15210b4877b85eca352f460558: Bootstrap starting.
I20260808 22:32:33.066321 16031 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b590ee15210b4877b85eca352f460558: Neither blocks nor log segments found. Creating new log.
I20260808 22:32:33.067189 16031 log.cc:824] T 00000000000000000000000000000000 P b590ee15210b4877b85eca352f460558: Log is configured to *not* fsync() on all Append() calls
I20260808 22:32:33.069473 16031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b590ee15210b4877b85eca352f460558: No bootstrap required, opened a new log
Aug 08 22:32:33 dist-test-slave-266d krb5kdc[15907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228352, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:32:33.073875 16031 raft_consensus.cc:348] T 00000000000000000000000000000000 P b590ee15210b4877b85eca352f460558 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b590ee15210b4877b85eca352f460558" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 46251 } }
I20260808 22:32:33.074239 16031 raft_consensus.cc:374] T 00000000000000000000000000000000 P b590ee15210b4877b85eca352f460558 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:32:33.074313 16031 raft_consensus.cc:729] T 00000000000000000000000000000000 P b590ee15210b4877b85eca352f460558 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b590ee15210b4877b85eca352f460558, State: Initialized, Role: FOLLOWER
I20260808 22:32:33.074800 16031 consensus_queue.cc:260] T 00000000000000000000000000000000 P b590ee15210b4877b85eca352f460558 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b590ee15210b4877b85eca352f460558" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 46251 } }
I20260808 22:32:33.074981 16031 raft_consensus.cc:388] T 00000000000000000000000000000000 P b590ee15210b4877b85eca352f460558 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:32:33.075052 16031 raft_consensus.cc:482] T 00000000000000000000000000000000 P b590ee15210b4877b85eca352f460558 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:32:33.075163 16031 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b590ee15210b4877b85eca352f460558 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:32:33.075521 16037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:33.061191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57742 (local address 127.31.87.190:46251)
0808 22:32:33.061636 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:33.061656 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:33.061699 (+ 43us) server_negotiation.cc:407] Connection header received
0808 22:32:33.062620 (+ 921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:33.062644 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:33.063046 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:33.063393 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:33.064108 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:33.065611 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:33.066493 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:33.067176 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:33.067339 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:33.070225 (+ 2886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:33.070233 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:33.070249 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:33.070284 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:33.072239 (+ 1955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:33.072695 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:33.072702 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:33.072708 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:33.072861 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:33.073152 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:33.073155 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:33.073157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:33.073694 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:33.073928 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:33.074120 (+ 192us) server_negotiation.cc:299] Negotiation successful
0808 22:32:33.074504 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":110,"threads_started":1}
I20260808 22:32:33.076368 16031 raft_consensus.cc:504] T 00000000000000000000000000000000 P b590ee15210b4877b85eca352f460558 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b590ee15210b4877b85eca352f460558" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 46251 } }
I20260808 22:32:33.076735 16031 leader_election.cc:302] T 00000000000000000000000000000000 P b590ee15210b4877b85eca352f460558 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b590ee15210b4877b85eca352f460558; no voters:
I20260808 22:32:33.077126 16031 leader_election.cc:288] T 00000000000000000000000000000000 P b590ee15210b4877b85eca352f460558 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:32:33.077237 16040 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b590ee15210b4877b85eca352f460558 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:32:33.077432 16040 raft_consensus.cc:686] T 00000000000000000000000000000000 P b590ee15210b4877b85eca352f460558 [term 1 LEADER]: Becoming Leader. State: Replica: b590ee15210b4877b85eca352f460558, State: Running, Role: LEADER
I20260808 22:32:33.077787 16040 consensus_queue.cc:237] T 00000000000000000000000000000000 P b590ee15210b4877b85eca352f460558 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b590ee15210b4877b85eca352f460558" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 46251 } }
I20260808 22:32:33.078464 16031 sys_catalog.cc:564] T 00000000000000000000000000000000 P b590ee15210b4877b85eca352f460558 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:32:33.078889 16040 sys_catalog.cc:455] T 00000000000000000000000000000000 P b590ee15210b4877b85eca352f460558 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b590ee15210b4877b85eca352f460558. Latest consensus state: current_term: 1 leader_uuid: "b590ee15210b4877b85eca352f460558" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b590ee15210b4877b85eca352f460558" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 46251 } } }
I20260808 22:32:33.079001 16040 sys_catalog.cc:458] T 00000000000000000000000000000000 P b590ee15210b4877b85eca352f460558 [sys.catalog]: This master's current role is: LEADER
I20260808 22:32:33.079288 16041 sys_catalog.cc:455] T 00000000000000000000000000000000 P b590ee15210b4877b85eca352f460558 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b590ee15210b4877b85eca352f460558" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b590ee15210b4877b85eca352f460558" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 46251 } } }
I20260808 22:32:33.079481 16041 sys_catalog.cc:458] T 00000000000000000000000000000000 P b590ee15210b4877b85eca352f460558 [sys.catalog]: This master's current role is: LEADER
I20260808 22:32:33.079581 16047 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:32:33.082300 16047 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:32:33.087316 16047 catalog_manager.cc:1347] Generated new cluster ID: dc2cd08e656743b9ad5b78f7386a525d
I20260808 22:32:33.087379 16047 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:32:33.101758 16047 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:32:33.102618 16047 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:32:33.114959 16047 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b590ee15210b4877b85eca352f460558: Generated new TSK 0
I20260808 22:32:33.115602 16047 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.87.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab.
Entry for principal kudu/127.31.87.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab.
I20260808 22:32:33.145889 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.129:0
--local_ip_for_outbound_sockets=127.31.87.129
--webserver_interface=127.31.87.129
--webserver_port=0
--tserver_master_addrs=127.31.87.190:46251
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab
--principal=kudu/127.31.87.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:36035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:32:33.252835 16068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:32:33.253070 16068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:32:33.253135 16068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:32:33.257021 16068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:32:33.257072 16068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:32:33.257165 16068 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.129
I20260808 22:32:33.262130 16068 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:36035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab
--principal=kudu/127.31.87.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.87.129
--webserver_port=0
--tserver_master_addrs=127.31.87.190:46251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16068
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.129
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:32:33.263228 16068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:32:33.264173 16068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:32:33.273280 16074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:33.273404 16079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:33.273304 16068 server_base.cc:1034] running on GCE node
W20260808 22:32:33.273306 16075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:33.273967 16068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:32:33.274510 16068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:32:33.275676 16068 hybrid_clock.cc:648] HybridClock initialized: now 1786228353275661 us; error 27 us; skew 500 ppm
Aug 08 22:32:33 dist-test-slave-266d krb5kdc[15907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228353, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:32:33.278254 16068 init.cc:377] Logged in from keytab as kudu/127.31.87.129@KRBTEST.COM (short username kudu)
I20260808 22:32:33.279138 16068 webserver.cc:460] Webserver started at http://127.31.87.129:38603/ using document root <none> and password file <none>
I20260808 22:32:33.279662 16068 fs_manager.cc:359] Metadata directory not provided
I20260808 22:32:33.279723 16068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:32:33.279889 16068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:32:33.281827 16068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/ts-0/data/instance:
uuid: "e3109e79013c4402a9ef2dcdd10d9d22"
format_stamp: "Formatted at 2026-08-08 22:32:33 on dist-test-slave-266d"
server_key: "866fbf8ab57524bc7eb88baf34966fea"
server_key_iv: "d5be5321c4d1b71b9868d73fe06bb3c8"
server_key_version: "encryptionkey@0"
I20260808 22:32:33.282323 16068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/ts-0/wal/instance:
uuid: "e3109e79013c4402a9ef2dcdd10d9d22"
format_stamp: "Formatted at 2026-08-08 22:32:33 on dist-test-slave-266d"
server_key: "866fbf8ab57524bc7eb88baf34966fea"
server_key_iv: "d5be5321c4d1b71b9868d73fe06bb3c8"
server_key_version: "encryptionkey@0"
I20260808 22:32:33.285741 16068 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:32:33.287868 16097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:33.288925 16068 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:32:33.289016 16068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/ts-0/wal
uuid: "e3109e79013c4402a9ef2dcdd10d9d22"
format_stamp: "Formatted at 2026-08-08 22:32:33 on dist-test-slave-266d"
server_key: "866fbf8ab57524bc7eb88baf34966fea"
server_key_iv: "d5be5321c4d1b71b9868d73fe06bb3c8"
server_key_version: "encryptionkey@0"
I20260808 22:32:33.289121 16068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:32:33.304677 16068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:32:33.306977 16068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:32:33.307106 16068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:32:33.307564 16068 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:32:33.308235 16068 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:32:33.308285 16068 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:33.308320 16068 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:32:33.308341 16068 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:33.316444 16068 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.129:34875
I20260808 22:32:33.316493 16279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.129:34875 every 8 connection(s)
I20260808 22:32:33.317380 16068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
I20260808 22:32:33.321336 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 16068
I20260808 22:32:33.321465 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/ts-0/wal/instance
I20260808 22:32:33.321727 32094 external_mini_cluster.cc:1428] Setting key ac4595a09f5f0e965492a1851ebc45c0
Aug 08 22:32:33 dist-test-slave-266d krb5kdc[15907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228353, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.129@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:32:33.330578 16037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:33.319012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:33503 (local address 127.31.87.190:46251)
0808 22:32:33.319144 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:33.319149 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:33.319934 (+ 785us) server_negotiation.cc:407] Connection header received
0808 22:32:33.321268 (+ 1334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:33.321271 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:33.321331 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:33.321424 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:33.322646 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:33.323314 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:33.324763 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:33.325011 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:33.325138 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:33.327332 (+ 2194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:33.327336 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:33.327339 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:33.327365 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:33.328644 (+ 1279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:33.329169 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:33.329173 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:33.329175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:33.329253 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:33.329569 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:33.329573 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:33.329574 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:33.329741 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:33.329827 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:33.330291 (+ 464us) server_negotiation.cc:299] Negotiation successful
0808 22:32:33.330418 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
WARNING: no policy specified for kudu/127.31.87.130@KRBTEST.COM; defaulting to no policy
I20260808 22:32:33.331295 16283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:33.319380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:46251 (local address 127.31.87.129:33503)
0808 22:32:33.319796 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:33.319836 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:33.321023 (+ 1187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:33.321644 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:33.321651 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:33.321993 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:33.322507 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:33.322522 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:33.323441 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:33.323445 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:33.324642 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:33.324653 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:33.325176 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:33.325187 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:33.325337 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:33.325832 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:33.325848 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:33.327181 (+ 1333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:32:33.328752 (+ 1571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:33.328758 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:33.328771 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:33.329040 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:33.329335 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:33.329338 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:33.329341 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:33.329468 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:33.329838 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:32:33.329845 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:32:33.330062 (+ 217us) client_negotiation.cc:769] Sending connection context
0808 22:32:33.330211 (+ 149us) client_negotiation.cc:240] Negotiation successful
0808 22:32:33.330506 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":108,"threads_started":1}
I20260808 22:32:33.332388 16280 heartbeater.cc:347] Connected to a master server at 127.31.87.190:46251
I20260808 22:32:33.332582 16280 heartbeater.cc:464] Registering TS with master...
I20260808 22:32:33.333007 16280 heartbeater.cc:511] Master 127.31.87.190:46251 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.130@KRBTEST.COM" created.
I20260808 22:32:33.334548 15969 ts_manager.cc:194] Registered new tserver with Master: e3109e79013c4402a9ef2dcdd10d9d22 (127.31.87.129:34875)
I20260808 22:32:33.335817 15969 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.129@KRBTEST.COM'} at 127.31.87.129:33503
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab.
Entry for principal kudu/127.31.87.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab.
I20260808 22:32:33.347570 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.130:0
--local_ip_for_outbound_sockets=127.31.87.130
--webserver_interface=127.31.87.130
--webserver_port=0
--tserver_master_addrs=127.31.87.190:46251
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab
--principal=kudu/127.31.87.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:36035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:32:33.455444 16289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:32:33.455699 16289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:32:33.455749 16289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:32:33.459697 16289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:32:33.459754 16289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:32:33.459811 16289 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.130
I20260808 22:32:33.464869 16289 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:36035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab
--principal=kudu/127.31.87.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.87.130
--webserver_port=0
--tserver_master_addrs=127.31.87.190:46251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16289
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.130
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:32:33.465850 16289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:32:33.466809 16289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:32:33.476465 16298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:33.476475 16297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:33.476691 16289 server_base.cc:1034] running on GCE node
W20260808 22:32:33.476480 16300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:33.477249 16289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:32:33.477869 16289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:32:33.479022 16289 hybrid_clock.cc:648] HybridClock initialized: now 1786228353478997 us; error 34 us; skew 500 ppm
Aug 08 22:32:33 dist-test-slave-266d krb5kdc[15907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228353, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:32:33.481706 16289 init.cc:377] Logged in from keytab as kudu/127.31.87.130@KRBTEST.COM (short username kudu)
I20260808 22:32:33.482548 16289 webserver.cc:460] Webserver started at http://127.31.87.130:38873/ using document root <none> and password file <none>
I20260808 22:32:33.483083 16289 fs_manager.cc:359] Metadata directory not provided
I20260808 22:32:33.483145 16289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:32:33.483332 16289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:32:33.485332 16289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/ts-1/data/instance:
uuid: "1b1628d38c484752aeecd3060e54f9e6"
format_stamp: "Formatted at 2026-08-08 22:32:33 on dist-test-slave-266d"
server_key: "a4d3b17753fe7bbbbb75194fe30a3865"
server_key_iv: "9635620469781793700e3aee2a5aa5f0"
server_key_version: "encryptionkey@0"
I20260808 22:32:33.485855 16289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/ts-1/wal/instance:
uuid: "1b1628d38c484752aeecd3060e54f9e6"
format_stamp: "Formatted at 2026-08-08 22:32:33 on dist-test-slave-266d"
server_key: "a4d3b17753fe7bbbbb75194fe30a3865"
server_key_iv: "9635620469781793700e3aee2a5aa5f0"
server_key_version: "encryptionkey@0"
I20260808 22:32:33.489482 16289 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 22:32:33.491731 16312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:33.492872 16289 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:32:33.492997 16289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/ts-1/wal
uuid: "1b1628d38c484752aeecd3060e54f9e6"
format_stamp: "Formatted at 2026-08-08 22:32:33 on dist-test-slave-266d"
server_key: "a4d3b17753fe7bbbbb75194fe30a3865"
server_key_iv: "9635620469781793700e3aee2a5aa5f0"
server_key_version: "encryptionkey@0"
I20260808 22:32:33.493105 16289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:32:33.506250 16289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:32:33.508819 16289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:32:33.508980 16289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:32:33.509577 16289 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:32:33.510371 16289 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:32:33.510417 16289 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:33.510468 16289 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:32:33.510494 16289 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:33.518714 16289 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.130:44497
I20260808 22:32:33.518740 16481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.130:44497 every 8 connection(s)
I20260808 22:32:33.519510 16289 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
I20260808 22:32:33.523239 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 16289
I20260808 22:32:33.523340 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/ts-1/wal/instance
I20260808 22:32:33.523653 32094 external_mini_cluster.cc:1428] Setting key 8ef99b5d79d45191915f3365c920124f
Aug 08 22:32:33 dist-test-slave-266d krb5kdc[15907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228353, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.130@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:32:33.532985 16037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:33.521155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:52445 (local address 127.31.87.190:46251)
0808 22:32:33.521277 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:33.521280 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:33.522060 (+ 780us) server_negotiation.cc:407] Connection header received
0808 22:32:33.523411 (+ 1351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:33.523415 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:33.523478 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:33.523571 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:33.524743 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:33.525415 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:33.526886 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:33.527122 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:33.527215 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:33.529651 (+ 2436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:33.529654 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:33.529656 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:33.529680 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:33.530991 (+ 1311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:33.531552 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:33.531554 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:33.531555 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:33.531622 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:33.531954 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:33.531956 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:33.531957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:33.532114 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:33.532197 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:33.532691 (+ 494us) server_negotiation.cc:299] Negotiation successful
0808 22:32:33.532826 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
WARNING: no policy specified for kudu/127.31.87.131@KRBTEST.COM; defaulting to no policy
I20260808 22:32:33.533824 16485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:33.521487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:46251 (local address 127.31.87.130:52445)
0808 22:32:33.521921 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:33.521960 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:33.523184 (+ 1224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:33.523721 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:33.523730 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:33.524129 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:33.524615 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:33.524628 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:33.525545 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:33.525548 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:33.526779 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:33.526792 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:33.527287 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:33.527292 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:33.527445 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:33.527976 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:33.527991 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:33.529488 (+ 1497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:32:33.531106 (+ 1618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:33.531113 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:33.531128 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:33.531461 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:33.531698 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:33.531701 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:33.531703 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:33.531876 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:33.532224 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:32:33.532230 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:32:33.532463 (+ 233us) client_negotiation.cc:769] Sending connection context
0808 22:32:33.532625 (+ 162us) client_negotiation.cc:240] Negotiation successful
0808 22:32:33.532928 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":108,"threads_started":1}
I20260808 22:32:33.534940 16482 heartbeater.cc:347] Connected to a master server at 127.31.87.190:46251
I20260808 22:32:33.535215 16482 heartbeater.cc:464] Registering TS with master...
I20260808 22:32:33.535785 16482 heartbeater.cc:511] Master 127.31.87.190:46251 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.131@KRBTEST.COM" created.
I20260808 22:32:33.537034 15969 ts_manager.cc:194] Registered new tserver with Master: 1b1628d38c484752aeecd3060e54f9e6 (127.31.87.130:44497)
I20260808 22:32:33.537599 15969 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.130@KRBTEST.COM'} at 127.31.87.130:52445
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab.
Entry for principal kudu/127.31.87.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab.
I20260808 22:32:33.550120 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.131:0
--local_ip_for_outbound_sockets=127.31.87.131
--webserver_interface=127.31.87.131
--webserver_port=0
--tserver_master_addrs=127.31.87.190:46251
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab
--principal=kudu/127.31.87.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:36035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:32:33.659485 16489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:32:33.659721 16489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:32:33.659760 16489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:32:33.663606 16489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:32:33.663654 16489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:32:33.663707 16489 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.131
I20260808 22:32:33.668666 16489 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:36035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab
--principal=kudu/127.31.87.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.87.131
--webserver_port=0
--tserver_master_addrs=127.31.87.190:46251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.131
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:32:33.669548 16489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:32:33.670428 16489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:32:33.679548 16497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:33.679577 16501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:32:33.679560 16499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:32:33.679775 16489 server_base.cc:1034] running on GCE node
I20260808 22:32:33.680272 16489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:32:33.680848 16489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:32:33.681998 16489 hybrid_clock.cc:648] HybridClock initialized: now 1786228353681967 us; error 42 us; skew 500 ppm
Aug 08 22:32:33 dist-test-slave-266d krb5kdc[15907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228353, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:32:33.684525 16489 init.cc:377] Logged in from keytab as kudu/127.31.87.131@KRBTEST.COM (short username kudu)
I20260808 22:32:33.685343 16489 webserver.cc:460] Webserver started at http://127.31.87.131:35461/ using document root <none> and password file <none>
I20260808 22:32:33.685848 16489 fs_manager.cc:359] Metadata directory not provided
I20260808 22:32:33.685911 16489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:32:33.686092 16489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:32:33.687969 16489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/ts-2/data/instance:
uuid: "4519da65fc9746819aef97377b93feee"
format_stamp: "Formatted at 2026-08-08 22:32:33 on dist-test-slave-266d"
server_key: "4d8dfcb7d3b2b7e66eb4c8f3497b468b"
server_key_iv: "b001c37e08d7c9f637101a01ef3dbdd0"
server_key_version: "encryptionkey@0"
I20260808 22:32:33.688482 16489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/ts-2/wal/instance:
uuid: "4519da65fc9746819aef97377b93feee"
format_stamp: "Formatted at 2026-08-08 22:32:33 on dist-test-slave-266d"
server_key: "4d8dfcb7d3b2b7e66eb4c8f3497b468b"
server_key_iv: "b001c37e08d7c9f637101a01ef3dbdd0"
server_key_version: "encryptionkey@0"
I20260808 22:32:33.691922 16489 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 22:32:33.694177 16511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:33.695237 16489 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:32:33.695354 16489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/ts-2/wal
uuid: "4519da65fc9746819aef97377b93feee"
format_stamp: "Formatted at 2026-08-08 22:32:33 on dist-test-slave-266d"
server_key: "4d8dfcb7d3b2b7e66eb4c8f3497b468b"
server_key_iv: "b001c37e08d7c9f637101a01ef3dbdd0"
server_key_version: "encryptionkey@0"
I20260808 22:32:33.695439 16489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:32:33.711601 16489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:32:33.714516 16489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:32:33.714685 16489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:32:33.715219 16489 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:32:33.715984 16489 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:32:33.716030 16489 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:33.716079 16489 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:32:33.716109 16489 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:32:33.724342 16489 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.131:45509
I20260808 22:32:33.724375 16672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.131:45509 every 8 connection(s)
I20260808 22:32:33.725190 16489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
I20260808 22:32:33.726011 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 16489
I20260808 22:32:33.726111 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/ts-2/wal/instance
I20260808 22:32:33.726380 32094 external_mini_cluster.cc:1428] Setting key 67a7d69df9989dcc449ee2d963516ca1
Aug 08 22:32:33 dist-test-slave-266d krb5kdc[15907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228353, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.131@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:32:33.738194 16037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:33.727526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:32969 (local address 127.31.87.190:46251)
0808 22:32:33.727648 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:33.727652 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:33.728616 (+ 964us) server_negotiation.cc:407] Connection header received
0808 22:32:33.729978 (+ 1362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:33.729981 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:33.730035 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:33.730104 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:33.731193 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:33.731736 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:33.732821 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:33.733035 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:33.733163 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:33.735380 (+ 2217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:33.735384 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:33.735386 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:33.735410 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:33.736558 (+ 1148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:33.736977 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:33.736979 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:33.736981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:33.737042 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:33.737335 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:33.737337 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:33.737338 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:33.737494 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:33.737604 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:33.737992 (+ 388us) server_negotiation.cc:299] Negotiation successful
0808 22:32:33.738080 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 22:32:33.738965 16676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:33.727985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:46251 (local address 127.31.87.131:32969)
0808 22:32:33.728470 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:33.728524 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:33.729792 (+ 1268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:33.730281 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:33.730287 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:33.730618 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:32:33.731063 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:33.731074 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:33.731848 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:33.731851 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:33.732697 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:33.732706 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:33.733192 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:33.733197 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:33.733367 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:33.733847 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:32:33.733863 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:32:33.735237 (+ 1374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:32:33.736654 (+ 1417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:33.736659 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:33.736669 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:33.736904 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:33.737138 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:32:33.737140 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:32:33.737141 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:32:33.737265 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:32:33.737598 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:32:33.737603 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:32:33.737812 (+ 209us) client_negotiation.cc:769] Sending connection context
0808 22:32:33.737948 (+ 136us) client_negotiation.cc:240] Negotiation successful
0808 22:32:33.738186 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":125,"threads_started":1}
I20260808 22:32:33.739991 16674 heartbeater.cc:347] Connected to a master server at 127.31.87.190:46251
I20260808 22:32:33.740217 16674 heartbeater.cc:464] Registering TS with master...
I20260808 22:32:33.740636 16674 heartbeater.cc:511] Master 127.31.87.190:46251 requested a full tablet report, sending...
I20260808 22:32:33.741554 15969 ts_manager.cc:194] Registered new tserver with Master: 4519da65fc9746819aef97377b93feee (127.31.87.131:45509)
I20260808 22:32:33.742074 15969 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.131@KRBTEST.COM'} at 127.31.87.131:32969
I20260808 22:32:33.751672 32094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:32:33.759773 16037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:33.753523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57756 (local address 127.31.87.190:46251)
0808 22:32:33.753659 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:33.753664 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:33.753809 (+ 145us) server_negotiation.cc:407] Connection header received
0808 22:32:33.753897 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:33.753899 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:33.753937 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:33.753998 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:32:33.754583 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:33.755090 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:33.755836 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:33.756036 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:33.756120 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:33.757303 (+ 1183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:32:33.757306 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:32:33.757308 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:32:33.757335 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:32:33.758558 (+ 1223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:33.758888 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:33.758890 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:33.758891 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:33.758956 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:32:33.759119 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:32:33.759120 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:32:33.759121 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:32:33.759304 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:32:33.759398 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:33.759541 (+ 143us) server_negotiation.cc:299] Negotiation successful
0808 22:32:33.759626 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 22:32:33.762336 15969 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57756:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:32:33.765295 15969 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:32:33.778674 16690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:33.773187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.130:44497 (local address 127.0.0.1:44754)
0808 22:32:33.773514 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:33.773564 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:33.773716 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:33.774287 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:33.774294 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:33.774319 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:32:33.774454 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:33.774461 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:33.775789 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:33.775793 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:33.777046 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:33.777057 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:33.778129 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:33.778133 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:33.778264 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:33.778308 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 22:32:33.778411 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 22:32:33.778525 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":68,"threads_started":1}
I20260808 22:32:33.778698 16692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:33.773299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44754 (local address 127.31.87.130:44497)
0808 22:32:33.773677 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:33.773689 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:33.773714 (+ 25us) server_negotiation.cc:407] Connection header received
0808 22:32:33.773882 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:33.773888 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:33.774099 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:33.774610 (+ 511us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:32:33.774773 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:33.775662 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:33.777206 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:33.778010 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:33.778126 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:33.778325 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:33.778442 (+ 117us) server_negotiation.cc:299] Negotiation successful
0808 22:32:33.778536 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":156,"threads_started":1}
I20260808 22:32:33.779000 16694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:33.773886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:34875 (local address 127.0.0.1:36840)
0808 22:32:33.774122 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:33.774135 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:33.774219 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:33.774896 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:33.774899 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:33.774914 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:32:33.774991 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:33.774996 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:33.776243 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:33.776246 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:33.777556 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:33.777568 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:33.778628 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:33.778632 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:33.778720 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:33.778745 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 22:32:33.778795 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 22:32:33.778876 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":80,"threads_started":1}
I20260808 22:32:33.779798 16695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:33.774061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36840 (local address 127.31.87.129:34875)
0808 22:32:33.774443 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:33.774454 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:33.774475 (+ 21us) server_negotiation.cc:407] Connection header received
0808 22:32:33.774547 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:33.774552 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:33.774750 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:33.775104 (+ 354us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:32:33.775230 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:33.776107 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:33.777677 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:33.778487 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:33.779375 (+ 888us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:33.779509 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:33.779556 (+ 47us) server_negotiation.cc:299] Negotiation successful
0808 22:32:33.779650 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":132,"threads_started":1}
I20260808 22:32:33.781503 16698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:33.774186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.131:45509 (local address 127.0.0.1:55470)
0808 22:32:33.775868 (+ 1682us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:33.775878 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:33.775958 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:33.776347 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:33.776350 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:33.776369 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:32:33.776506 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:33.776512 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:33.777854 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:33.777858 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:33.778850 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:33.778862 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:33.779869 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:33.779874 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:33.781198 (+ 1324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:33.781225 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 22:32:33.781313 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 22:32:33.781379 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1629,"mutex_wait_us":23,"thread_start_us":84,"threads_started":1}
I20260808 22:32:33.781697 16699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:33.774674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55470 (local address 127.31.87.131:45509)
0808 22:32:33.775068 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:33.775078 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:33.775966 (+ 888us) server_negotiation.cc:407] Connection header received
0808 22:32:33.776049 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:33.776054 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:33.776187 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:33.776347 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:32:33.776614 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:33.777735 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:33.778986 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:33.779741 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:33.779845 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:33.779970 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:33.781356 (+ 1386us) server_negotiation.cc:299] Negotiation successful
0808 22:32:33.781455 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":105,"threads_started":1}
I20260808 22:32:33.782114 16384 tablet_service.cc:1467] Processing CreateTablet for tablet c69d5a3b115a4d5fab341a5ed71409d1 (DEFAULT_TABLE table=test-table [id=5e24f3c7136544c4b36a561df1165db6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:32:33.782353 16174 tablet_service.cc:1467] Processing CreateTablet for tablet c69d5a3b115a4d5fab341a5ed71409d1 (DEFAULT_TABLE table=test-table [id=5e24f3c7136544c4b36a561df1165db6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:32:33.783071 16384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c69d5a3b115a4d5fab341a5ed71409d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:32:33.783242 16174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c69d5a3b115a4d5fab341a5ed71409d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:32:33.786938 16580 tablet_service.cc:1467] Processing CreateTablet for tablet c69d5a3b115a4d5fab341a5ed71409d1 (DEFAULT_TABLE table=test-table [id=5e24f3c7136544c4b36a561df1165db6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:32:33.787981 16580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c69d5a3b115a4d5fab341a5ed71409d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:32:33.789762 16700 tablet_bootstrap.cc:492] T c69d5a3b115a4d5fab341a5ed71409d1 P e3109e79013c4402a9ef2dcdd10d9d22: Bootstrap starting.
I20260808 22:32:33.790539 16704 tablet_bootstrap.cc:492] T c69d5a3b115a4d5fab341a5ed71409d1 P 1b1628d38c484752aeecd3060e54f9e6: Bootstrap starting.
I20260808 22:32:33.791785 16700 tablet_bootstrap.cc:654] T c69d5a3b115a4d5fab341a5ed71409d1 P e3109e79013c4402a9ef2dcdd10d9d22: Neither blocks nor log segments found. Creating new log.
I20260808 22:32:33.792919 16700 log.cc:824] T c69d5a3b115a4d5fab341a5ed71409d1 P e3109e79013c4402a9ef2dcdd10d9d22: Log is configured to *not* fsync() on all Append() calls
I20260808 22:32:33.793229 16704 tablet_bootstrap.cc:654] T c69d5a3b115a4d5fab341a5ed71409d1 P 1b1628d38c484752aeecd3060e54f9e6: Neither blocks nor log segments found. Creating new log.
I20260808 22:32:33.794040 16704 log.cc:824] T c69d5a3b115a4d5fab341a5ed71409d1 P 1b1628d38c484752aeecd3060e54f9e6: Log is configured to *not* fsync() on all Append() calls
I20260808 22:32:33.794709 16700 tablet_bootstrap.cc:492] T c69d5a3b115a4d5fab341a5ed71409d1 P e3109e79013c4402a9ef2dcdd10d9d22: No bootstrap required, opened a new log
I20260808 22:32:33.794896 16700 ts_tablet_manager.cc:1397] T c69d5a3b115a4d5fab341a5ed71409d1 P e3109e79013c4402a9ef2dcdd10d9d22: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 22:32:33.795861 16709 tablet_bootstrap.cc:492] T c69d5a3b115a4d5fab341a5ed71409d1 P 4519da65fc9746819aef97377b93feee: Bootstrap starting.
I20260808 22:32:33.795958 16704 tablet_bootstrap.cc:492] T c69d5a3b115a4d5fab341a5ed71409d1 P 1b1628d38c484752aeecd3060e54f9e6: No bootstrap required, opened a new log
I20260808 22:32:33.796130 16704 ts_tablet_manager.cc:1397] T c69d5a3b115a4d5fab341a5ed71409d1 P 1b1628d38c484752aeecd3060e54f9e6: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 22:32:33.798049 16709 tablet_bootstrap.cc:654] T c69d5a3b115a4d5fab341a5ed71409d1 P 4519da65fc9746819aef97377b93feee: Neither blocks nor log segments found. Creating new log.
I20260808 22:32:33.798192 16700 raft_consensus.cc:348] T c69d5a3b115a4d5fab341a5ed71409d1 P e3109e79013c4402a9ef2dcdd10d9d22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b1628d38c484752aeecd3060e54f9e6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 44497 } } peers { permanent_uuid: "e3109e79013c4402a9ef2dcdd10d9d22" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 34875 } } peers { permanent_uuid: "4519da65fc9746819aef97377b93feee" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45509 } }
I20260808 22:32:33.798406 16700 raft_consensus.cc:374] T c69d5a3b115a4d5fab341a5ed71409d1 P e3109e79013c4402a9ef2dcdd10d9d22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:32:33.798458 16700 raft_consensus.cc:729] T c69d5a3b115a4d5fab341a5ed71409d1 P e3109e79013c4402a9ef2dcdd10d9d22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3109e79013c4402a9ef2dcdd10d9d22, State: Initialized, Role: FOLLOWER
I20260808 22:32:33.798940 16700 consensus_queue.cc:260] T c69d5a3b115a4d5fab341a5ed71409d1 P e3109e79013c4402a9ef2dcdd10d9d22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b1628d38c484752aeecd3060e54f9e6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 44497 } } peers { permanent_uuid: "e3109e79013c4402a9ef2dcdd10d9d22" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 34875 } } peers { permanent_uuid: "4519da65fc9746819aef97377b93feee" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45509 } }
I20260808 22:32:33.799566 16280 heartbeater.cc:503] Master 127.31.87.190:46251 was elected leader, sending a full tablet report...
I20260808 22:32:33.799564 16709 log.cc:824] T c69d5a3b115a4d5fab341a5ed71409d1 P 4519da65fc9746819aef97377b93feee: Log is configured to *not* fsync() on all Append() calls
I20260808 22:32:33.799861 16704 raft_consensus.cc:348] T c69d5a3b115a4d5fab341a5ed71409d1 P 1b1628d38c484752aeecd3060e54f9e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b1628d38c484752aeecd3060e54f9e6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 44497 } } peers { permanent_uuid: "e3109e79013c4402a9ef2dcdd10d9d22" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 34875 } } peers { permanent_uuid: "4519da65fc9746819aef97377b93feee" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45509 } }
I20260808 22:32:33.800094 16704 raft_consensus.cc:374] T c69d5a3b115a4d5fab341a5ed71409d1 P 1b1628d38c484752aeecd3060e54f9e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:32:33.800148 16704 raft_consensus.cc:729] T c69d5a3b115a4d5fab341a5ed71409d1 P 1b1628d38c484752aeecd3060e54f9e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b1628d38c484752aeecd3060e54f9e6, State: Initialized, Role: FOLLOWER
I20260808 22:32:33.800549 16704 consensus_queue.cc:260] T c69d5a3b115a4d5fab341a5ed71409d1 P 1b1628d38c484752aeecd3060e54f9e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b1628d38c484752aeecd3060e54f9e6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 44497 } } peers { permanent_uuid: "e3109e79013c4402a9ef2dcdd10d9d22" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 34875 } } peers { permanent_uuid: "4519da65fc9746819aef97377b93feee" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45509 } }
I20260808 22:32:33.801011 16700 ts_tablet_manager.cc:1428] T c69d5a3b115a4d5fab341a5ed71409d1 P e3109e79013c4402a9ef2dcdd10d9d22: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 22:32:33.801493 16704 ts_tablet_manager.cc:1428] T c69d5a3b115a4d5fab341a5ed71409d1 P 1b1628d38c484752aeecd3060e54f9e6: Time spent starting tablet: real 0.005s user 0.001s sys 0.005s
I20260808 22:32:33.801801 16709 tablet_bootstrap.cc:492] T c69d5a3b115a4d5fab341a5ed71409d1 P 4519da65fc9746819aef97377b93feee: No bootstrap required, opened a new log
I20260808 22:32:33.801849 16482 heartbeater.cc:503] Master 127.31.87.190:46251 was elected leader, sending a full tablet report...
I20260808 22:32:33.801985 16709 ts_tablet_manager.cc:1397] T c69d5a3b115a4d5fab341a5ed71409d1 P 4519da65fc9746819aef97377b93feee: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 22:32:33.805333 16709 raft_consensus.cc:348] T c69d5a3b115a4d5fab341a5ed71409d1 P 4519da65fc9746819aef97377b93feee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b1628d38c484752aeecd3060e54f9e6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 44497 } } peers { permanent_uuid: "e3109e79013c4402a9ef2dcdd10d9d22" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 34875 } } peers { permanent_uuid: "4519da65fc9746819aef97377b93feee" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45509 } }
I20260808 22:32:33.805516 16709 raft_consensus.cc:374] T c69d5a3b115a4d5fab341a5ed71409d1 P 4519da65fc9746819aef97377b93feee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:32:33.805572 16709 raft_consensus.cc:729] T c69d5a3b115a4d5fab341a5ed71409d1 P 4519da65fc9746819aef97377b93feee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4519da65fc9746819aef97377b93feee, State: Initialized, Role: FOLLOWER
I20260808 22:32:33.805907 16709 consensus_queue.cc:260] T c69d5a3b115a4d5fab341a5ed71409d1 P 4519da65fc9746819aef97377b93feee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b1628d38c484752aeecd3060e54f9e6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 44497 } } peers { permanent_uuid: "e3109e79013c4402a9ef2dcdd10d9d22" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 34875 } } peers { permanent_uuid: "4519da65fc9746819aef97377b93feee" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45509 } }
I20260808 22:32:33.806437 16674 heartbeater.cc:503] Master 127.31.87.190:46251 was elected leader, sending a full tablet report...
I20260808 22:32:33.806720 16709 ts_tablet_manager.cc:1428] T c69d5a3b115a4d5fab341a5ed71409d1 P 4519da65fc9746819aef97377b93feee: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260808 22:32:33.818464 16282 tablet.cc:2369] T c69d5a3b115a4d5fab341a5ed71409d1 P e3109e79013c4402a9ef2dcdd10d9d22: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 22:32:33.976073 16675 tablet.cc:2369] T c69d5a3b115a4d5fab341a5ed71409d1 P 4519da65fc9746819aef97377b93feee: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 22:32:34.020604 16484 tablet.cc:2369] T c69d5a3b115a4d5fab341a5ed71409d1 P 1b1628d38c484752aeecd3060e54f9e6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:32:34.083632 16715 raft_consensus.cc:482] T c69d5a3b115a4d5fab341a5ed71409d1 P 4519da65fc9746819aef97377b93feee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:32:34.083830 16715 raft_consensus.cc:504] T c69d5a3b115a4d5fab341a5ed71409d1 P 4519da65fc9746819aef97377b93feee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b1628d38c484752aeecd3060e54f9e6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 44497 } } peers { permanent_uuid: "e3109e79013c4402a9ef2dcdd10d9d22" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 34875 } } peers { permanent_uuid: "4519da65fc9746819aef97377b93feee" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45509 } }
I20260808 22:32:34.084903 16715 leader_election.cc:288] T c69d5a3b115a4d5fab341a5ed71409d1 P 4519da65fc9746819aef97377b93feee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b1628d38c484752aeecd3060e54f9e6 (127.31.87.130:44497), e3109e79013c4402a9ef2dcdd10d9d22 (127.31.87.129:34875)
I20260808 22:32:34.089329 16692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:34.085345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:46099 (local address 127.31.87.130:44497)
0808 22:32:34.085479 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:34.085484 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:34.085499 (+ 15us) server_negotiation.cc:407] Connection header received
0808 22:32:34.085626 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:34.085630 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:34.085681 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:34.085755 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:32:34.086072 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:34.086785 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:34.088234 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:34.088782 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:34.088896 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:34.088949 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:34.089139 (+ 190us) server_negotiation.cc:299] Negotiation successful
0808 22:32:34.089203 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 22:32:34.089341 16676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:34.085267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.130:44497 (local address 127.31.87.131:46099)
0808 22:32:34.085391 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:34.085408 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:34.085514 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:34.085794 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:34.085798 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:34.085823 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:32:34.085952 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:34.085959 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:34.086895 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:34.086897 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:34.088115 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:34.088127 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:34.088897 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:34.088900 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:34.088988 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:34.089014 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 22:32:34.089126 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 22:32:34.089205 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":26}
I20260808 22:32:34.089480 16695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:34.085710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:35751 (local address 127.31.87.129:34875)
0808 22:32:34.085839 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:34.085843 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:34.086104 (+ 261us) server_negotiation.cc:407] Connection header received
0808 22:32:34.086194 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:34.086197 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:34.086246 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:34.086332 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:32:34.086513 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:34.087098 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:34.088396 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:34.088941 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:34.089038 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:34.089074 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:34.089310 (+ 236us) server_negotiation.cc:299] Negotiation successful
0808 22:32:34.089376 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 22:32:34.089560 16718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:34.085558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:34875 (local address 127.31.87.131:35751)
0808 22:32:34.086015 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:34.086038 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 22:32:34.086123 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:32:34.086335 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:32:34.086337 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:32:34.086348 (+ 11us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:32:34.086437 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:34.086441 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:34.087191 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:34.087194 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:34.088305 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:32:34.088313 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:32:34.089046 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:32:34.089049 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:32:34.089170 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:34.089191 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 22:32:34.089260 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 22:32:34.089315 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"mutex_wait_us":19,"thread_start_us":138,"threads_started":1}
I20260808 22:32:34.090042 16419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c69d5a3b115a4d5fab341a5ed71409d1" candidate_uuid: "4519da65fc9746819aef97377b93feee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b1628d38c484752aeecd3060e54f9e6" is_pre_election: true
I20260808 22:32:34.090086 16211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c69d5a3b115a4d5fab341a5ed71409d1" candidate_uuid: "4519da65fc9746819aef97377b93feee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3109e79013c4402a9ef2dcdd10d9d22" is_pre_election: true
I20260808 22:32:34.090322 16419 raft_consensus.cc:2454] T c69d5a3b115a4d5fab341a5ed71409d1 P 1b1628d38c484752aeecd3060e54f9e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4519da65fc9746819aef97377b93feee in term 0.
I20260808 22:32:34.090334 16211 raft_consensus.cc:2454] T c69d5a3b115a4d5fab341a5ed71409d1 P e3109e79013c4402a9ef2dcdd10d9d22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4519da65fc9746819aef97377b93feee in term 0.
I20260808 22:32:34.090737 16514 leader_election.cc:302] T c69d5a3b115a4d5fab341a5ed71409d1 P 4519da65fc9746819aef97377b93feee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b1628d38c484752aeecd3060e54f9e6, 4519da65fc9746819aef97377b93feee; no voters:
I20260808 22:32:34.090974 16715 raft_consensus.cc:2781] T c69d5a3b115a4d5fab341a5ed71409d1 P 4519da65fc9746819aef97377b93feee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:32:34.091049 16715 raft_consensus.cc:482] T c69d5a3b115a4d5fab341a5ed71409d1 P 4519da65fc9746819aef97377b93feee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:32:34.091078 16715 raft_consensus.cc:3037] T c69d5a3b115a4d5fab341a5ed71409d1 P 4519da65fc9746819aef97377b93feee [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:32:34.092128 16715 raft_consensus.cc:504] T c69d5a3b115a4d5fab341a5ed71409d1 P 4519da65fc9746819aef97377b93feee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b1628d38c484752aeecd3060e54f9e6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 44497 } } peers { permanent_uuid: "e3109e79013c4402a9ef2dcdd10d9d22" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 34875 } } peers { permanent_uuid: "4519da65fc9746819aef97377b93feee" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45509 } }
I20260808 22:32:34.092420 16715 leader_election.cc:288] T c69d5a3b115a4d5fab341a5ed71409d1 P 4519da65fc9746819aef97377b93feee [CANDIDATE]: Term 1 election: Requested vote from peers 1b1628d38c484752aeecd3060e54f9e6 (127.31.87.130:44497), e3109e79013c4402a9ef2dcdd10d9d22 (127.31.87.129:34875)
I20260808 22:32:34.092777 16419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c69d5a3b115a4d5fab341a5ed71409d1" candidate_uuid: "4519da65fc9746819aef97377b93feee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b1628d38c484752aeecd3060e54f9e6"
I20260808 22:32:34.092831 16211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c69d5a3b115a4d5fab341a5ed71409d1" candidate_uuid: "4519da65fc9746819aef97377b93feee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3109e79013c4402a9ef2dcdd10d9d22"
I20260808 22:32:34.092883 16419 raft_consensus.cc:3037] T c69d5a3b115a4d5fab341a5ed71409d1 P 1b1628d38c484752aeecd3060e54f9e6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:32:34.092947 16211 raft_consensus.cc:3037] T c69d5a3b115a4d5fab341a5ed71409d1 P e3109e79013c4402a9ef2dcdd10d9d22 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:32:34.093896 16419 raft_consensus.cc:2454] T c69d5a3b115a4d5fab341a5ed71409d1 P 1b1628d38c484752aeecd3060e54f9e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4519da65fc9746819aef97377b93feee in term 1.
I20260808 22:32:34.094033 16211 raft_consensus.cc:2454] T c69d5a3b115a4d5fab341a5ed71409d1 P e3109e79013c4402a9ef2dcdd10d9d22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4519da65fc9746819aef97377b93feee in term 1.
I20260808 22:32:34.094157 16514 leader_election.cc:302] T c69d5a3b115a4d5fab341a5ed71409d1 P 4519da65fc9746819aef97377b93feee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b1628d38c484752aeecd3060e54f9e6, 4519da65fc9746819aef97377b93feee; no voters:
I20260808 22:32:34.094318 16715 raft_consensus.cc:2781] T c69d5a3b115a4d5fab341a5ed71409d1 P 4519da65fc9746819aef97377b93feee [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:32:34.094518 16715 raft_consensus.cc:686] T c69d5a3b115a4d5fab341a5ed71409d1 P 4519da65fc9746819aef97377b93feee [term 1 LEADER]: Becoming Leader. State: Replica: 4519da65fc9746819aef97377b93feee, State: Running, Role: LEADER
I20260808 22:32:34.094842 16715 consensus_queue.cc:237] T c69d5a3b115a4d5fab341a5ed71409d1 P 4519da65fc9746819aef97377b93feee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b1628d38c484752aeecd3060e54f9e6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 44497 } } peers { permanent_uuid: "e3109e79013c4402a9ef2dcdd10d9d22" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 34875 } } peers { permanent_uuid: "4519da65fc9746819aef97377b93feee" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45509 } }
I20260808 22:32:34.097813 15968 catalog_manager.cc:5495] T c69d5a3b115a4d5fab341a5ed71409d1 P 4519da65fc9746819aef97377b93feee reported cstate change: term changed from 0 to 1, leader changed from <none> to 4519da65fc9746819aef97377b93feee (127.31.87.131). New cstate: current_term: 1 leader_uuid: "4519da65fc9746819aef97377b93feee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b1628d38c484752aeecd3060e54f9e6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 44497 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e3109e79013c4402a9ef2dcdd10d9d22" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 34875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4519da65fc9746819aef97377b93feee" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 45509 } health_report { overall_health: HEALTHY } } }
I20260808 22:32:34.139637 16699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:34.136600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55482 (local address 127.31.87.131:45509)
0808 22:32:34.136728 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 22:32:34.136731 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:32:34.136758 (+ 27us) server_negotiation.cc:407] Connection header received
0808 22:32:34.136827 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:32:34.136829 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:32:34.136881 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:32:34.136958 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:32:34.137181 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:34.137660 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:34.138440 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:32:34.138663 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:32:34.138741 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:32:34.138946 (+ 205us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:32:34.139339 (+ 393us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:32:34.139423 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 22:32:34.139466 (+ 43us) server_negotiation.cc:299] Negotiation successful
0808 22:32:34.139512 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 22:32:34.145113 16211 raft_consensus.cc:1261] T c69d5a3b115a4d5fab341a5ed71409d1 P e3109e79013c4402a9ef2dcdd10d9d22 [term 1 FOLLOWER]: Refusing update from remote peer 4519da65fc9746819aef97377b93feee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:32:34.145124 16419 raft_consensus.cc:1261] T c69d5a3b115a4d5fab341a5ed71409d1 P 1b1628d38c484752aeecd3060e54f9e6 [term 1 FOLLOWER]: Refusing update from remote peer 4519da65fc9746819aef97377b93feee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:32:34.145856 16719 consensus_queue.cc:1035] T c69d5a3b115a4d5fab341a5ed71409d1 P 4519da65fc9746819aef97377b93feee [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b1628d38c484752aeecd3060e54f9e6" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 44497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:32:34.146027 16715 consensus_queue.cc:1035] T c69d5a3b115a4d5fab341a5ed71409d1 P 4519da65fc9746819aef97377b93feee [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3109e79013c4402a9ef2dcdd10d9d22" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 34875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:32:34.154450 16725 mvcc.cc:205] Tried to move back new op lower bound from 7316391338571292672 to 7316391338374791168. Current Snapshot: MvccSnapshot[applied={T|T < 7316391338571292672}]
I20260808 22:32:34.161280 15968 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57756:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:32:34.161453 15968 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57756:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:32:34.163718 15968 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b590ee15210b4877b85eca352f460558: Sending DeleteTablet for 3 replicas of tablet c69d5a3b115a4d5fab341a5ed71409d1
I20260808 22:32:34.164357 16384 tablet_service.cc:1514] Processing DeleteTablet for tablet c69d5a3b115a4d5fab341a5ed71409d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:32:34 UTC) from {username='kudu', principal='kudu/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:44754
I20260808 22:32:34.164438 16174 tablet_service.cc:1514] Processing DeleteTablet for tablet c69d5a3b115a4d5fab341a5ed71409d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:32:34 UTC) from {username='kudu', principal='kudu/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:36840
I20260808 22:32:34.164539 16580 tablet_service.cc:1514] Processing DeleteTablet for tablet c69d5a3b115a4d5fab341a5ed71409d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:32:34 UTC) from {username='kudu', principal='kudu/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:55470
I20260808 22:32:34.164908 16737 tablet_replica.cc:330] T c69d5a3b115a4d5fab341a5ed71409d1 P 1b1628d38c484752aeecd3060e54f9e6: stopping tablet replica
I20260808 22:32:34.165192 16739 tablet_replica.cc:330] T c69d5a3b115a4d5fab341a5ed71409d1 P 4519da65fc9746819aef97377b93feee: stopping tablet replica
I20260808 22:32:34.165333 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 16068
I20260808 22:32:34.171644 16739 raft_consensus.cc:2229] T c69d5a3b115a4d5fab341a5ed71409d1 P 4519da65fc9746819aef97377b93feee [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:32:34.172104 16739 raft_consensus.cc:2258] T c69d5a3b115a4d5fab341a5ed71409d1 P 4519da65fc9746819aef97377b93feee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:32:34.175563 16739 ts_tablet_manager.cc:1905] T c69d5a3b115a4d5fab341a5ed71409d1 P 4519da65fc9746819aef97377b93feee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:32:34.177920 16737 raft_consensus.cc:2229] T c69d5a3b115a4d5fab341a5ed71409d1 P 1b1628d38c484752aeecd3060e54f9e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:32:34.178174 16739 ts_tablet_manager.cc:1918] T c69d5a3b115a4d5fab341a5ed71409d1 P 4519da65fc9746819aef97377b93feee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:32:34.178258 16739 log.cc:1196] T c69d5a3b115a4d5fab341a5ed71409d1 P 4519da65fc9746819aef97377b93feee: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/ts-2/wal/wals/c69d5a3b115a4d5fab341a5ed71409d1
I20260808 22:32:34.178296 16737 raft_consensus.cc:2258] T c69d5a3b115a4d5fab341a5ed71409d1 P 1b1628d38c484752aeecd3060e54f9e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:32:34.178699 16739 ts_tablet_manager.cc:1939] T c69d5a3b115a4d5fab341a5ed71409d1 P 4519da65fc9746819aef97377b93feee: Deleting consensus metadata
I20260808 22:32:34.179895 15949 catalog_manager.cc:4841] TS 4519da65fc9746819aef97377b93feee (127.31.87.131:45509): tablet c69d5a3b115a4d5fab341a5ed71409d1 (table test-table [id=5e24f3c7136544c4b36a561df1165db6]) successfully deleted
I20260808 22:32:34.180339 16737 ts_tablet_manager.cc:1905] T c69d5a3b115a4d5fab341a5ed71409d1 P 1b1628d38c484752aeecd3060e54f9e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:32:34.183096 16737 ts_tablet_manager.cc:1918] T c69d5a3b115a4d5fab341a5ed71409d1 P 1b1628d38c484752aeecd3060e54f9e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:32:34.183202 16737 log.cc:1196] T c69d5a3b115a4d5fab341a5ed71409d1 P 1b1628d38c484752aeecd3060e54f9e6: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786228291013016-32094-0/minicluster-data/ts-1/wal/wals/c69d5a3b115a4d5fab341a5ed71409d1
I20260808 22:32:34.183501 16737 ts_tablet_manager.cc:1939] T c69d5a3b115a4d5fab341a5ed71409d1 P 1b1628d38c484752aeecd3060e54f9e6: Deleting consensus metadata
W20260808 22:32:34.184146 15949 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.87.129:34875 (error 108)
I20260808 22:32:34.184612 15949 catalog_manager.cc:4841] TS 1b1628d38c484752aeecd3060e54f9e6 (127.31.87.130:44497): tablet c69d5a3b115a4d5fab341a5ed71409d1 (table test-table [id=5e24f3c7136544c4b36a561df1165db6]) successfully deleted
I20260808 22:32:34.185142 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 16289
I20260808 22:32:34.185173 16698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:32:34.184879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:34875 (local address 127.0.0.1:36846)
0808 22:32:34.185000 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 22:32:34.185065 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.129:34875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 22:32:34.194116 15949 catalog_manager.cc:4584] TS e3109e79013c4402a9ef2dcdd10d9d22 (127.31.87.129:34875): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c69d5a3b115a4d5fab341a5ed71409d1: Network error: Client connection negotiation failed: client connection to 127.31.87.129:34875: connect: Connection refused (error 111)
I20260808 22:32:34.200356 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 16489
I20260808 22:32:34.211421 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 15923
2026-08-08T22:32:34Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2603 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:32:34 dist-test-slave-266d krb5kdc[16747](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:32:34 dist-test-slave-266d krb5kdc[16747](info): set up 2 sockets
Aug 08 22:32:34 dist-test-slave-266d krb5kdc[16747](info): commencing operation
krb5kdc: starting...
W20260808 22:32:35.352494 32094 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Aug 08 22:32:35 dist-test-slave-266d krb5kdc[16747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:32:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:32:35Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.31.87.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.31.87.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.31.87.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/krb5kdc/rangeradmin_127.31.87.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.31.87.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/krb5kdc/rangeradmin_127.31.87.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.31.87.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.31.87.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.31.87.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/krb5kdc/rangerlookup_127.31.87.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.31.87.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/krb5kdc/rangerlookup_127.31.87.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.31.87.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.31.87.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.31.87.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/krb5kdc/HTTP_127.31.87.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.31.87.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/krb5kdc/HTTP_127.31.87.148@KRBTEST.COM.keytab.
I20260808 22:32:35.509760 32094 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskOEAG_G/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/postgres -l logfile start
2026-08-08 22:32:37.771 UTC [16800] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 22:32:37.771 UTC [16800] LOG: listening on IPv4 address "127.31.87.148", port 48755
2026-08-08 22:32:37.774 UTC [16800] LOG: listening on Unix socket "/tmp/.s.PGSQL.48755"
2026-08-08 22:32:37.794 UTC [16805] LOG: database system was shut down at 2026-08-08 22:32:36 UTC
2026-08-08 22:32:37.797 UTC [16800] LOG: database system is ready to accept connections
I20260808 22:32:38.857969 32094 mini_postgres.cc:96] Postgres bound to 48755
2026-08-08 22:32:38.866 UTC [16818] FATAL: database "slave" does not exist
127.31.87.148:48755 - accepting connections
I20260808 22:32:38.866881 32094 mini_ranger.cc:161] Starting Ranger...
I20260808 22:32:38.882414 32094 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 22:32:39.281673 32094 mini_ranger.cc:87] Created ranger Postgres database
I20260808 22:32:39.281757 32094 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ranger-admin
2026-08-08 22:32:39,588 [I] DB FLAVOR :POSTGRES
2026-08-08 22:32:39,588 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 22:32:39,588 [I] Checking connection..
2026-08-08 22:32:39,588 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOEAG_G/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOEAG_G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.87.148:48755/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 22:32:39,848 [I] Checking connection passed.
2026-08-08 22:32:39,848 [I] --------- Verifying version history table ---------
2026-08-08 22:32:39,849 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOEAG_G/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOEAG_G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.87.148:48755/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 22:32:40,086 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 22:32:40,086 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOEAG_G/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOEAG_G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.87.148:48755/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 22:32:40,340 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 22:32:40,340 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 22:32:40,340 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOEAG_G/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOEAG_G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.87.148:48755/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskOEAG_G/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 22:32:40.588 UTC [16919] WARNING: there is no transaction in progress
2026-08-08 22:32:40,596 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 22:32:40,596 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOEAG_G/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOEAG_G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.87.148:48755/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 22:32:40,852 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 22:32:40,852 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 22:32:40,853 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOEAG_G/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOEAG_G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.87.148:48755/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 22:32:41,094 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOEAG_G/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOEAG_G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.87.148:48755/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 22:32:41,343 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOEAG_G/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOEAG_G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.87.148:48755/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-266d.c.gcp-upstream.internal','N') ;"
2026-08-08 22:32:41,568 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOEAG_G/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOEAG_G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.87.148:48755/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 22:32:41,824 [I] Table x_portal_user does not exist in database ranger
2026-08-08 22:32:41,825 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOEAG_G/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOEAG_G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.87.148:48755/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 22:32:42,080 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 22:32:42,080 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOEAG_G/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOEAG_G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.87.148:48755/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 22:32:42,318 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOEAG_G/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOEAG_G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.87.148:48755/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 22:32:42,570 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 22:32:42,570 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOEAG_G/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOEAG_G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.87.148:48755/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskOEAG_G/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 22:32:43.450 UTC [17076] WARNING: there is no transaction in progress
2026-08-08 22:32:43.465 UTC [17076] WARNING: there is no transaction in progress
2026-08-08 22:32:43.479 UTC [17076] WARNING: there is no transaction in progress
2026-08-08 22:32:43.492 UTC [17076] WARNING: there is no transaction in progress
2026-08-08 22:32:43.506 UTC [17076] WARNING: there is no transaction in progress
2026-08-08 22:32:43.523 UTC [17076] WARNING: there is no transaction in progress
2026-08-08 22:32:43.573 UTC [17076] WARNING: there is no transaction in progress
2026-08-08 22:32:43.579 UTC [17076] WARNING: there is no transaction in progress
2026-08-08 22:32:43.592 UTC [17076] WARNING: there is no transaction in progress
2026-08-08 22:32:43.602 UTC [17076] WARNING: there is no transaction in progress
2026-08-08 22:32:43.612 UTC [17076] WARNING: there is no transaction in progress
2026-08-08 22:32:43.625 UTC [17076] WARNING: there is no transaction in progress
2026-08-08 22:32:43.633 UTC [17076] WARNING: there is no transaction in progress
2026-08-08 22:32:43.639 UTC [17076] WARNING: there is no transaction in progress
2026-08-08 22:32:43.645 UTC [17076] WARNING: there is no transaction in progress
2026-08-08 22:32:43.650 UTC [17076] WARNING: there is no transaction in progress
2026-08-08 22:32:44,188 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 22:32:44,188 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOEAG_G/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOEAG_G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.87.148:48755/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 22:32:44,439 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 22:32:44,440 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOEAG_G/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOEAG_G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.87.148:48755/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 22:32:44,661 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 22:32:44,662 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOEAG_G/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOEAG_G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.87.148:48755/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 22:32:44,916 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 22:32:44,916 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOEAG_G/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOEAG_G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.87.148:48755/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 22:32:45,168 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOEAG_G/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOEAG_G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.87.148:48755/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 22:32:45,416 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOEAG_G/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOEAG_G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.87.148:48755/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-266d.c.gcp-upstream.internal';"
2026-08-08 22:32:45,640 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 22:32:45,640 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOEAG_G/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOEAG_G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.87.148:48755/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 22:32:45,896 [I] DB_PATCHES have already been applied
I20260808 22:32:45.902117 32094 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ranger-admin:/tmp/dist-test-taskOEAG_G/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskOEAG_G/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskOEAG_G/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOEAG_G/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 22:32:45.902185 32094 mini_ranger.cc:193] Using host: 127.31.87.148
I20260808 22:32:45.906535 32094 mini_ranger.cc:227] Ranger admin URL: http://127.31.87.148:53677
Aug 08, 2026 10:32:46 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 10:32:46 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 10:32:46 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 10:32:46 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 10:32:46 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 10:32:46 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 10:32:46 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 22:32:47.116372 32094 mini_ranger.cc:160] Time spent starting Ranger: real 8.250s user 0.006s sys 0.006s
Aug 08, 2026 10:33:05 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 10:33:05 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 08, 2026 10:33:05 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 10:33:05 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 10:33:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 10:33:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 10:33:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 10:33:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 10:33:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 10:33:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 10:33:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 10:33:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 10:33:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 10:33:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 10:33:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 10:33:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 10:33:05 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 10:33:06 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 22:33:07.202872 32094 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.31.87.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.31.87.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.31.87.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/krb5kdc/rangerkms_127.31.87.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.31.87.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/krb5kdc/rangerkms_127.31.87.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.31.87.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.31.87.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.31.87.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/krb5kdc/HTTP_127.31.87.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.31.87.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/krb5kdc/HTTP_127.31.87.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 22:33:07 dist-test-slave-266d krb5kdc[16747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228387, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 22:33:07.282840 32094 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 22:33:07.292600 32094 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 22:33:08.014583 32094 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 22:33:08.014685 32094 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ranger-kms
I20260808 22:33:08.014796 32094 mini_ranger_kms.cc:227] Using postgres at 127.31.87.148:48755
2026-08-08 22:33:08,602 [I] DB FLAVOR :POSTGRES
2026-08-08 22:33:08,602 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 22:33:08,602 [I] Checking connection
2026-08-08 22:33:08,603 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOEAG_G/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.87.148:48755/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 22:33:08,836 [I] connection success
2026-08-08 22:33:08,837 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 22:33:08,837 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 22:33:08,837 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOEAG_G/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.87.148:48755/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 22:33:09,063 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 22:33:09,063 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 22:33:09,064 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 22:33:09,064 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOEAG_G/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.87.148:48755/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 22:33:09,321 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 22:33:09.773437 32094 mini_ranger_kms.cc:310] Created kms service
I20260808 22:33:09.954224 32094 mini_ranger_kms.cc:326] Created kudu user
I20260808 22:33:09.995582 32094 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 22:33:09 dist-test-slave-266d krb5kdc[16747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228387, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.31.87.148@KRBTEST.COM
I20260808 22:33:10.376022 32094 mini_ranger_kms.cc:384] Added ranger policy
I20260808 22:33:10.376192 32094 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ranger-kms:/tmp/dist-test-taskOEAG_G/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOEAG_G/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskOEAG_G/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 22:33:10.376219 32094 mini_ranger_kms.cc:240] Using host: 127.31.87.148
I20260808 22:33:10.381419 32094 mini_ranger_kms.cc:276] Ranger KMS PID: 17353
I20260808 22:33:10.381541 32094 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.31.87.148:45547
Aug 08, 2026 10:33:11 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 10:33:11 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 10:33:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 10:33:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 10:33:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 10:33:11 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 22:33:12 dist-test-slave-266d krb5kdc[16747](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786228392, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.87.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:33:12.819689 32094 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.537s user 0.000s sys 0.281s
I20260808 22:33:12.819809 32094 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 22:33:12.819921 32094 mini_ranger_kms.cc:401] 127.31.87.148:45547/kms/v1/keys
Aug 08 22:33:13 dist-test-slave-266d krb5kdc[16747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228392, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.87.148@KRBTEST.COM for HTTP/127.31.87.148@KRBTEST.COM
Aug 08 22:33:13 dist-test-slave-266d krb5kdc[16747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228392, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.87.148@KRBTEST.COM for HTTP/127.31.87.148@KRBTEST.COM
Aug 08 22:33:15 dist-test-slave-266d krb5kdc[16747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.31.87.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
I20260808 22:33:15.735837 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:40057
--webserver_interface=127.31.87.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:43533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:40057
--ranger_config_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.87.148:45547/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:33:15.849915 17427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:33:15.850157 17427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:33:15.850195 17427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:33:15.854256 17427 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:33:15.854310 17427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:33:15.854328 17427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:33:15.854342 17427 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:33:15.854360 17427 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:33:15.859768 17427 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:43533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/master-0/wal
--ranger_kms_url=127.31.87.148:45547/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:40057
--ranger_config_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:40057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.87.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17427
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:33:15.860791 17427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:33:15.861830 17427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:33:15.870587 17441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:33:15.870733 17437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:33:15.870819 17427 server_base.cc:1034] running on GCE node
W20260808 22:33:15.870592 17436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:33:15.871650 17427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:33:15.872988 17427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:33:15.874185 17427 hybrid_clock.cc:648] HybridClock initialized: now 1786228395874153 us; error 40 us; skew 500 ppm
Aug 08 22:33:15 dist-test-slave-266d krb5kdc[16747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228395, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:33:15.880240 17427 init.cc:377] Logged in from keytab as kudu/127.31.87.190@KRBTEST.COM (short username kudu)
I20260808 22:33:15.881413 17427 webserver.cc:460] Webserver started at http://127.31.87.190:38469/ using document root <none> and password file <none>
I20260808 22:33:15.882103 17427 fs_manager.cc:359] Metadata directory not provided
I20260808 22:33:15.882187 17427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:33:15.882433 17427 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 22:33:15 dist-test-slave-266d krb5kdc[16747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228395, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for HTTP/127.31.87.148@KRBTEST.COM
I20260808 22:33:15.936228 17427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/master-0/data/instance:
uuid: "936307bb66fd4309a8a3c78fe120d1c4"
format_stamp: "Formatted at 2026-08-08 22:33:15 on dist-test-slave-266d"
server_key: "fcd44808005ec34bf859cc159ae80315"
server_key_iv: "eb537e94b59bf626ded946c6243b2b70"
server_key_version: "kuduclusterkey@0"
I20260808 22:33:15.936987 17427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/master-0/wal/instance:
uuid: "936307bb66fd4309a8a3c78fe120d1c4"
format_stamp: "Formatted at 2026-08-08 22:33:15 on dist-test-slave-266d"
server_key: "fcd44808005ec34bf859cc159ae80315"
server_key_iv: "eb537e94b59bf626ded946c6243b2b70"
server_key_version: "kuduclusterkey@0"
I20260808 22:33:15.941982 17427 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260808 22:33:15.959513 17451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:33:15.961059 17427 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260808 22:33:15.961221 17427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "936307bb66fd4309a8a3c78fe120d1c4"
format_stamp: "Formatted at 2026-08-08 22:33:15 on dist-test-slave-266d"
server_key: "fcd44808005ec34bf859cc159ae80315"
server_key_iv: "eb537e94b59bf626ded946c6243b2b70"
server_key_version: "kuduclusterkey@0"
I20260808 22:33:15.961359 17427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:33:15.978533 17427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:33:15.981446 17427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:33:15.981632 17427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:33:15.990449 17427 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.190:40057
I20260808 22:33:15.990458 17529 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.190:40057 every 8 connection(s)
I20260808 22:33:15.991577 17427 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:33:15.992692 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 17427
I20260808 22:33:15.992834 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/master-0/wal/instance
I20260808 22:33:15.993180 32094 external_mini_cluster.cc:1428] Setting key d6fe62222a74e961d273e63fb0c2293f
I20260808 22:33:15.994863 17530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:33:16.001001 17530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 936307bb66fd4309a8a3c78fe120d1c4: Bootstrap starting.
I20260808 22:33:16.003546 17530 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 936307bb66fd4309a8a3c78fe120d1c4: Neither blocks nor log segments found. Creating new log.
Aug 08 22:33:15 dist-test-slave-266d krb5kdc[16747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:33:16.004551 17530 log.cc:824] T 00000000000000000000000000000000 P 936307bb66fd4309a8a3c78fe120d1c4: Log is configured to *not* fsync() on all Append() calls
I20260808 22:33:16.007535 17530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 936307bb66fd4309a8a3c78fe120d1c4: No bootstrap required, opened a new log
I20260808 22:33:16.010318 17535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:33:15.994548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50148 (local address 127.31.87.190:40057)
0808 22:33:15.995062 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0808 22:33:15.995081 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 22:33:15.995125 (+ 44us) server_negotiation.cc:407] Connection header received
0808 22:33:15.995970 (+ 845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:33:15.995992 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:33:15.996411 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:33:15.996772 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:33:15.997660 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:33:15.999204 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:33:16.000383 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:33:16.001085 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:33:16.001338 (+ 253us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:33:16.004235 (+ 2897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:33:16.004242 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:33:16.004257 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:33:16.004288 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:33:16.006670 (+ 2382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:33:16.007176 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:33:16.007182 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:33:16.007187 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:33:16.007315 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:33:16.007675 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:33:16.007678 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:33:16.007680 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:33:16.008205 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:33:16.008401 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0808 22:33:16.009076 (+ 675us) server_negotiation.cc:299] Negotiation successful
0808 22:33:16.009391 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":202,"threads_started":1}
I20260808 22:33:16.012066 17530 raft_consensus.cc:348] T 00000000000000000000000000000000 P 936307bb66fd4309a8a3c78fe120d1c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "936307bb66fd4309a8a3c78fe120d1c4" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 40057 } }
I20260808 22:33:16.012277 17530 raft_consensus.cc:374] T 00000000000000000000000000000000 P 936307bb66fd4309a8a3c78fe120d1c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:33:16.012360 17530 raft_consensus.cc:729] T 00000000000000000000000000000000 P 936307bb66fd4309a8a3c78fe120d1c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 936307bb66fd4309a8a3c78fe120d1c4, State: Initialized, Role: FOLLOWER
I20260808 22:33:16.012820 17530 consensus_queue.cc:260] T 00000000000000000000000000000000 P 936307bb66fd4309a8a3c78fe120d1c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "936307bb66fd4309a8a3c78fe120d1c4" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 40057 } }
I20260808 22:33:16.012976 17530 raft_consensus.cc:388] T 00000000000000000000000000000000 P 936307bb66fd4309a8a3c78fe120d1c4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:33:16.013041 17530 raft_consensus.cc:482] T 00000000000000000000000000000000 P 936307bb66fd4309a8a3c78fe120d1c4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:33:16.013159 17530 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 936307bb66fd4309a8a3c78fe120d1c4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:33:16.013984 17530 raft_consensus.cc:504] T 00000000000000000000000000000000 P 936307bb66fd4309a8a3c78fe120d1c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "936307bb66fd4309a8a3c78fe120d1c4" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 40057 } }
I20260808 22:33:16.014320 17530 leader_election.cc:302] T 00000000000000000000000000000000 P 936307bb66fd4309a8a3c78fe120d1c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 936307bb66fd4309a8a3c78fe120d1c4; no voters:
I20260808 22:33:16.014685 17530 leader_election.cc:288] T 00000000000000000000000000000000 P 936307bb66fd4309a8a3c78fe120d1c4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:33:16.014762 17538 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 936307bb66fd4309a8a3c78fe120d1c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:33:16.015009 17538 raft_consensus.cc:686] T 00000000000000000000000000000000 P 936307bb66fd4309a8a3c78fe120d1c4 [term 1 LEADER]: Becoming Leader. State: Replica: 936307bb66fd4309a8a3c78fe120d1c4, State: Running, Role: LEADER
I20260808 22:33:16.015367 17538 consensus_queue.cc:237] T 00000000000000000000000000000000 P 936307bb66fd4309a8a3c78fe120d1c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "936307bb66fd4309a8a3c78fe120d1c4" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 40057 } }
I20260808 22:33:16.015879 17530 sys_catalog.cc:564] T 00000000000000000000000000000000 P 936307bb66fd4309a8a3c78fe120d1c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:33:16.017004 17539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 936307bb66fd4309a8a3c78fe120d1c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "936307bb66fd4309a8a3c78fe120d1c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "936307bb66fd4309a8a3c78fe120d1c4" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 40057 } } }
I20260808 22:33:16.017150 17539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 936307bb66fd4309a8a3c78fe120d1c4 [sys.catalog]: This master's current role is: LEADER
I20260808 22:33:16.017410 17539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 936307bb66fd4309a8a3c78fe120d1c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 936307bb66fd4309a8a3c78fe120d1c4. Latest consensus state: current_term: 1 leader_uuid: "936307bb66fd4309a8a3c78fe120d1c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "936307bb66fd4309a8a3c78fe120d1c4" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 40057 } } }
I20260808 22:33:16.017491 17539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 936307bb66fd4309a8a3c78fe120d1c4 [sys.catalog]: This master's current role is: LEADER
I20260808 22:33:16.018092 17530 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 22:33:16.018729 17550 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:33:16.024816 17550 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:33:16.037376 17550 catalog_manager.cc:1347] Generated new cluster ID: 6de291c756f94e56baa1d37d589ef808
I20260808 22:33:16.037441 17550 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:33:16.048630 17550 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:33:16.049826 17550 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:33:16.069520 17550 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 936307bb66fd4309a8a3c78fe120d1c4: Generated new TSK 0
I20260808 22:33:16.070403 17550 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.87.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab.
Entry for principal kudu/127.31.87.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab.
I20260808 22:33:16.804618 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.87.148:45547/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.129:0
--local_ip_for_outbound_sockets=127.31.87.129
--webserver_interface=127.31.87.129
--webserver_port=0
--tserver_master_addrs=127.31.87.190:40057
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab
--principal=kudu/127.31.87.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:43533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:33:16.925619 17601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:33:16.925853 17601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:33:16.925892 17601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:33:16.929847 17601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:33:16.929905 17601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:33:16.929992 17601 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.129
I20260808 22:33:16.935381 17601 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:43533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.31.87.148:45547/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab
--principal=kudu/127.31.87.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.87.129
--webserver_port=0
--tserver_master_addrs=127.31.87.190:40057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17601
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.129
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:33:16.936465 17601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:33:16.937587 17601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:33:16.947172 17610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:33:16.947239 17612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:33:16.947439 17601 server_base.cc:1034] running on GCE node
W20260808 22:33:16.947265 17615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:33:16.947927 17601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:33:16.948549 17601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:33:16.949743 17601 hybrid_clock.cc:648] HybridClock initialized: now 1786228396949714 us; error 47 us; skew 500 ppm
Aug 08 22:33:16 dist-test-slave-266d krb5kdc[16747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228396, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:33:16.952456 17601 init.cc:377] Logged in from keytab as kudu/127.31.87.129@KRBTEST.COM (short username kudu)
I20260808 22:33:16.953531 17601 webserver.cc:460] Webserver started at http://127.31.87.129:39933/ using document root <none> and password file <none>
I20260808 22:33:16.954241 17601 fs_manager.cc:359] Metadata directory not provided
I20260808 22:33:16.954337 17601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:33:16.954589 17601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 22:33:16 dist-test-slave-266d krb5kdc[16747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228396, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.129@KRBTEST.COM for HTTP/127.31.87.148@KRBTEST.COM
I20260808 22:33:16.969794 17601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ts-0/data/instance:
uuid: "676e1dff3a2949508bb00bc94561220d"
format_stamp: "Formatted at 2026-08-08 22:33:16 on dist-test-slave-266d"
server_key: "5b5627867e87a69310e33550fc1d8cee"
server_key_iv: "991fc5ba4fa5d020e4dfc31fadc675c6"
server_key_version: "kuduclusterkey@0"
I20260808 22:33:16.970359 17601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ts-0/wal/instance:
uuid: "676e1dff3a2949508bb00bc94561220d"
format_stamp: "Formatted at 2026-08-08 22:33:16 on dist-test-slave-266d"
server_key: "5b5627867e87a69310e33550fc1d8cee"
server_key_iv: "991fc5ba4fa5d020e4dfc31fadc675c6"
server_key_version: "kuduclusterkey@0"
I20260808 22:33:16.974309 17601 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 22:33:16.993294 17626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:33:16.994802 17601 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 22:33:16.994923 17601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ts-0/wal
uuid: "676e1dff3a2949508bb00bc94561220d"
format_stamp: "Formatted at 2026-08-08 22:33:16 on dist-test-slave-266d"
server_key: "5b5627867e87a69310e33550fc1d8cee"
server_key_iv: "991fc5ba4fa5d020e4dfc31fadc675c6"
server_key_version: "kuduclusterkey@0"
I20260808 22:33:16.995036 17601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:33:17.018298 17601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:33:17.021399 17601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:33:17.021572 17601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:33:17.022128 17601 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:33:17.023056 17601 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:33:17.023103 17601 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:33:17.023157 17601 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:33:17.023211 17601 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:33:17.034293 17601 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.129:35795
I20260808 22:33:17.034420 17800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.129:35795 every 8 connection(s)
I20260808 22:33:17.035257 17601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
I20260808 22:33:17.042701 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 17601
I20260808 22:33:17.042833 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ts-0/wal/instance
I20260808 22:33:17.043162 32094 external_mini_cluster.cc:1428] Setting key 717c0dac54ad8cb93ac91f7ad637a6c4
Aug 08 22:33:17 dist-test-slave-266d krb5kdc[16747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228396, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.129@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:33:17.051869 17804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:33:17.037151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:37685 (local address 127.31.87.190:40057)
0808 22:33:17.037449 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0808 22:33:17.037458 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 22:33:17.038221 (+ 763us) server_negotiation.cc:407] Connection header received
0808 22:33:17.039178 (+ 957us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:33:17.039182 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:33:17.039241 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:33:17.039330 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:33:17.040373 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:33:17.040923 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:33:17.042116 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:33:17.042355 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:33:17.047609 (+ 5254us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:33:17.047746 (+ 137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:33:17.047750 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:33:17.047753 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:33:17.047782 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:33:17.049599 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:33:17.050171 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:33:17.050174 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:33:17.050176 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:33:17.050264 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:33:17.050628 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:33:17.050631 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:33:17.050632 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:33:17.050841 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:33:17.050916 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 22:33:17.051555 (+ 639us) server_negotiation.cc:299] Negotiation successful
0808 22:33:17.051680 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":110,"threads_started":1}
I20260808 22:33:17.053653 17806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:33:17.037621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:40057 (local address 127.31.87.129:37685)
0808 22:33:17.038072 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0808 22:33:17.038122 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 22:33:17.038975 (+ 853us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:33:17.039509 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:33:17.039516 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:33:17.039777 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:33:17.040228 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:33:17.040241 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:33:17.041038 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:33:17.041041 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:33:17.042001 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:33:17.042011 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:33:17.042492 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:33:17.042496 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:33:17.042686 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:33:17.043528 (+ 842us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:33:17.043547 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:33:17.045200 (+ 1653us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:33:17.049731 (+ 4531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:33:17.049737 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:33:17.049750 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:33:17.050057 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:33:17.050366 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:33:17.050369 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:33:17.050371 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:33:17.050525 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:33:17.050981 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:33:17.050987 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:33:17.051270 (+ 283us) client_negotiation.cc:769] Sending connection context
0808 22:33:17.052289 (+ 1019us) client_negotiation.cc:240] Negotiation successful
0808 22:33:17.052621 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":116,"threads_started":1}
I20260808 22:33:17.055497 17801 heartbeater.cc:347] Connected to a master server at 127.31.87.190:40057
I20260808 22:33:17.055717 17801 heartbeater.cc:464] Registering TS with master...
I20260808 22:33:17.056239 17801 heartbeater.cc:511] Master 127.31.87.190:40057 requested a full tablet report, sending...
I20260808 22:33:17.057925 17478 ts_manager.cc:194] Registered new tserver with Master: 676e1dff3a2949508bb00bc94561220d (127.31.87.129:35795)
I20260808 22:33:17.059415 17478 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.129@KRBTEST.COM'} at 127.31.87.129:37685
WARNING: no policy specified for kudu/127.31.87.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab.
Entry for principal kudu/127.31.87.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab.
I20260808 22:33:17.096247 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.87.148:45547/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.130:0
--local_ip_for_outbound_sockets=127.31.87.130
--webserver_interface=127.31.87.130
--webserver_port=0
--tserver_master_addrs=127.31.87.190:40057
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab
--principal=kudu/127.31.87.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:43533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 22:33:17 dist-test-slave-266d krb5kdc[16747](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786228397, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 22:33:17.256467 17811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:33:17.256789 17811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:33:17.256852 17811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:33:17.262823 17811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:33:17.262902 17811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:33:17.262987 17811 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.130
I20260808 22:33:17.270527 17811 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:43533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.31.87.148:45547/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab
--principal=kudu/127.31.87.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.87.130
--webserver_port=0
--tserver_master_addrs=127.31.87.190:40057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17811
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.130
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:33:17.271922 17811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:33:17.273167 17811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:33:17.286571 17824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:33:17.289430 17823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:33:17.289731 17834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:33:17.290473 17811 server_base.cc:1034] running on GCE node
I20260808 22:33:17.291039 17811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:33:17.291778 17811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:33:17.299551 17811 hybrid_clock.cc:648] HybridClock initialized: now 1786228397299475 us; error 86 us; skew 500 ppm
Aug 08 22:33:17 dist-test-slave-266d krb5kdc[16747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228397, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:33:17.303169 17811 init.cc:377] Logged in from keytab as kudu/127.31.87.130@KRBTEST.COM (short username kudu)
I20260808 22:33:17.304268 17811 webserver.cc:460] Webserver started at http://127.31.87.130:34863/ using document root <none> and password file <none>
I20260808 22:33:17.304999 17811 fs_manager.cc:359] Metadata directory not provided
I20260808 22:33:17.305138 17811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:33:17.305462 17811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 22:33:17 dist-test-slave-266d krb5kdc[16747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228397, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.130@KRBTEST.COM for HTTP/127.31.87.148@KRBTEST.COM
I20260808 22:33:17.322283 17811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ts-1/data/instance:
uuid: "32dde4ef77a44e6f8171b17f62dcbcc9"
format_stamp: "Formatted at 2026-08-08 22:33:17 on dist-test-slave-266d"
server_key: "81f1f5ec7d85524ae46510c28bd48f7b"
server_key_iv: "03018a83278e1272d4f5134da891ee9f"
server_key_version: "kuduclusterkey@0"
I20260808 22:33:17.324460 17811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ts-1/wal/instance:
uuid: "32dde4ef77a44e6f8171b17f62dcbcc9"
format_stamp: "Formatted at 2026-08-08 22:33:17 on dist-test-slave-266d"
server_key: "81f1f5ec7d85524ae46510c28bd48f7b"
server_key_iv: "03018a83278e1272d4f5134da891ee9f"
server_key_version: "kuduclusterkey@0"
I20260808 22:33:17.333724 17811 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.004s sys 0.000s
I20260808 22:33:17.353168 17845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:33:17.355072 17811 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260808 22:33:17.355213 17811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ts-1/wal
uuid: "32dde4ef77a44e6f8171b17f62dcbcc9"
format_stamp: "Formatted at 2026-08-08 22:33:17 on dist-test-slave-266d"
server_key: "81f1f5ec7d85524ae46510c28bd48f7b"
server_key_iv: "03018a83278e1272d4f5134da891ee9f"
server_key_version: "kuduclusterkey@0"
I20260808 22:33:17.355361 17811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:33:17.378234 17811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:33:17.381963 17811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:33:17.382223 17811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:33:17.382959 17811 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:33:17.384110 17811 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:33:17.384181 17811 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:33:17.384239 17811 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:33:17.384286 17811 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:33:17.396332 17811 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.130:36329
I20260808 22:33:17.396411 18024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.130:36329 every 8 connection(s)
I20260808 22:33:17.397663 17811 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
I20260808 22:33:17.398448 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 17811
I20260808 22:33:17.398574 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ts-1/wal/instance
I20260808 22:33:17.398887 32094 external_mini_cluster.cc:1428] Setting key abdbdfc657af7860ce4f3ae8a1fea551
Aug 08 22:33:17 dist-test-slave-266d krb5kdc[16747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228397, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.130@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.87.131@KRBTEST.COM; defaulting to no policy
I20260808 22:33:17.416529 17804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:33:17.400799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:52453 (local address 127.31.87.190:40057)
0808 22:33:17.400976 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 22:33:17.400981 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:33:17.402726 (+ 1745us) server_negotiation.cc:407] Connection header received
0808 22:33:17.404010 (+ 1284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:33:17.404013 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:33:17.404070 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:33:17.404127 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:33:17.405842 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:33:17.406653 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:33:17.408396 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:33:17.408757 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:33:17.408841 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:33:17.411912 (+ 3071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:33:17.411916 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:33:17.411920 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:33:17.411948 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:33:17.413674 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:33:17.414307 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:33:17.414311 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:33:17.414312 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:33:17.414401 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:33:17.414901 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:33:17.414906 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:33:17.414908 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:33:17.415198 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:33:17.415363 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0808 22:33:17.416080 (+ 717us) server_negotiation.cc:299] Negotiation successful
0808 22:33:17.416232 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 22:33:17.418363 18030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:33:17.401312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:40057 (local address 127.31.87.130:52453)
0808 22:33:17.402570 (+ 1258us) negotiation.cc:107] Waiting for socket to connect
0808 22:33:17.402627 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 22:33:17.403760 (+ 1133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:33:17.404561 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:33:17.404569 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:33:17.404941 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:33:17.405657 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:33:17.405678 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:33:17.406830 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:33:17.406833 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:33:17.408248 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:33:17.408261 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:33:17.409054 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:33:17.409058 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:33:17.409302 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:33:17.409924 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:33:17.409945 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:33:17.411701 (+ 1756us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:33:17.413838 (+ 2137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:33:17.413844 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:33:17.413860 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:33:17.414183 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:33:17.414558 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:33:17.414561 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:33:17.414563 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:33:17.414738 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:33:17.415426 (+ 688us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:33:17.415433 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:33:17.415755 (+ 322us) client_negotiation.cc:769] Sending connection context
0808 22:33:17.416870 (+ 1115us) client_negotiation.cc:240] Negotiation successful
0808 22:33:17.417296 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":992,"thread_start_us":132,"threads_started":1}
I20260808 22:33:17.424932 18026 heartbeater.cc:347] Connected to a master server at 127.31.87.190:40057
I20260808 22:33:17.425230 18026 heartbeater.cc:464] Registering TS with master...
I20260808 22:33:17.425915 18026 heartbeater.cc:511] Master 127.31.87.190:40057 requested a full tablet report, sending...
I20260808 22:33:17.427174 17478 ts_manager.cc:194] Registered new tserver with Master: 32dde4ef77a44e6f8171b17f62dcbcc9 (127.31.87.130:36329)
I20260808 22:33:17.427963 17478 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.130@KRBTEST.COM'} at 127.31.87.130:52453
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab.
Entry for principal kudu/127.31.87.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab.
I20260808 22:33:17.449638 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.87.148:45547/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.131:0
--local_ip_for_outbound_sockets=127.31.87.131
--webserver_interface=127.31.87.131
--webserver_port=0
--tserver_master_addrs=127.31.87.190:40057
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab
--principal=kudu/127.31.87.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:43533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:33:17.617156 18035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:33:17.617566 18035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:33:17.617697 18035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:33:17.623733 18035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:33:17.623862 18035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:33:17.624006 18035 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.131
I20260808 22:33:17.631640 18035 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:43533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.31.87.148:45547/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab
--principal=kudu/127.31.87.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.87.131
--webserver_port=0
--tserver_master_addrs=127.31.87.190:40057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18035
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.131
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
Aug 08 22:33:17 dist-test-slave-266d krb5kdc[16747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228397, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for HTTP/127.31.87.148@KRBTEST.COM
I20260808 22:33:17.634045 18035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:33:17.635432 18035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:33:17.648959 18044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:33:17.650701 18043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:33:17.649152 18046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:33:17.650314 18035 server_base.cc:1034] running on GCE node
I20260808 22:33:17.656200 18035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:33:17.657009 18035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:33:17.660283 18035 hybrid_clock.cc:648] HybridClock initialized: now 1786228397660211 us; error 85 us; skew 500 ppm
Aug 08 22:33:17 dist-test-slave-266d krb5kdc[16747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228397, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:33:17.664224 18035 init.cc:377] Logged in from keytab as kudu/127.31.87.131@KRBTEST.COM (short username kudu)
I20260808 22:33:17.665369 18035 webserver.cc:460] Webserver started at http://127.31.87.131:33139/ using document root <none> and password file <none>
I20260808 22:33:17.666098 18035 fs_manager.cc:359] Metadata directory not provided
I20260808 22:33:17.666186 18035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:33:17.666697 18035 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 22:33:17 dist-test-slave-266d krb5kdc[16747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228397, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.131@KRBTEST.COM for HTTP/127.31.87.148@KRBTEST.COM
I20260808 22:33:17.682381 18035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ts-2/data/instance:
uuid: "9e08615db4024e949d48130f199e847a"
format_stamp: "Formatted at 2026-08-08 22:33:17 on dist-test-slave-266d"
server_key: "e32383b26295da06109276891963b083"
server_key_iv: "fc74cb47fc0e504be135c7a7df1097a5"
server_key_version: "kuduclusterkey@0"
I20260808 22:33:17.684252 18035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ts-2/wal/instance:
uuid: "9e08615db4024e949d48130f199e847a"
format_stamp: "Formatted at 2026-08-08 22:33:17 on dist-test-slave-266d"
server_key: "e32383b26295da06109276891963b083"
server_key_iv: "fc74cb47fc0e504be135c7a7df1097a5"
server_key_version: "kuduclusterkey@0"
I20260808 22:33:17.690724 18035 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260808 22:33:17.720357 18054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:33:17.722282 18035 fs_manager.cc:725] Time spent opening block manager: real 0.015s user 0.003s sys 0.000s
I20260808 22:33:17.722434 18035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ts-2/wal
uuid: "9e08615db4024e949d48130f199e847a"
format_stamp: "Formatted at 2026-08-08 22:33:17 on dist-test-slave-266d"
server_key: "e32383b26295da06109276891963b083"
server_key_iv: "fc74cb47fc0e504be135c7a7df1097a5"
server_key_version: "kuduclusterkey@0"
I20260808 22:33:17.722604 18035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:33:17.760172 18035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:33:17.767343 18035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:33:17.767704 18035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:33:17.768709 18035 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:33:17.770008 18035 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:33:17.770154 18035 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:33:17.770270 18035 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:33:17.770354 18035 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:33:17.782850 18035 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.131:38045
I20260808 22:33:17.784132 18035 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
I20260808 22:33:17.785719 18231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.131:38045 every 8 connection(s)
I20260808 22:33:17.791131 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 18035
I20260808 22:33:17.791282 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ts-2/wal/instance
I20260808 22:33:17.791627 32094 external_mini_cluster.cc:1428] Setting key c909a99848bff02c3ab85ca333499aa9
Aug 08 22:33:17 dist-test-slave-266d krb5kdc[16747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228397, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.131@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:33:17.807289 17804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:33:17.787648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:55805 (local address 127.31.87.190:40057)
0808 22:33:17.787819 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 22:33:17.787824 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:33:17.788804 (+ 980us) server_negotiation.cc:407] Connection header received
0808 22:33:17.790263 (+ 1459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:33:17.790269 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:33:17.790331 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:33:17.790388 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:33:17.794297 (+ 3909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:33:17.795127 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:33:17.797004 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:33:17.797395 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:33:17.797481 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:33:17.800953 (+ 3472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:33:17.800958 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:33:17.800961 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:33:17.800991 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:33:17.802819 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:33:17.803506 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:33:17.803510 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:33:17.803511 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:33:17.803617 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:33:17.804083 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:33:17.804088 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:33:17.804090 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:33:17.804342 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:33:17.804421 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 22:33:17.806964 (+ 2543us) server_negotiation.cc:299] Negotiation successful
0808 22:33:17.807113 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 22:33:17.809310 18234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:33:17.788045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:40057 (local address 127.31.87.131:55805)
0808 22:33:17.788632 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0808 22:33:17.788688 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 22:33:17.789976 (+ 1288us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:33:17.790820 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:33:17.790830 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:33:17.791286 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:33:17.794109 (+ 2823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:33:17.794137 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:33:17.795339 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:33:17.795342 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:33:17.796844 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:33:17.796858 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:33:17.797712 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:33:17.797717 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:33:17.798003 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:33:17.798852 (+ 849us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:33:17.798877 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:33:17.800741 (+ 1864us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:33:17.802987 (+ 2246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:33:17.802993 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:33:17.803011 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:33:17.803375 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:33:17.803749 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:33:17.803752 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:33:17.803754 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:33:17.803950 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:33:17.806311 (+ 2361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:33:17.806319 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:33:17.806644 (+ 325us) client_negotiation.cc:769] Sending connection context
0808 22:33:17.807788 (+ 1144us) client_negotiation.cc:240] Negotiation successful
0808 22:33:17.808218 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":127,"threads_started":1}
I20260808 22:33:17.810824 18232 heartbeater.cc:347] Connected to a master server at 127.31.87.190:40057
I20260808 22:33:17.811081 18232 heartbeater.cc:464] Registering TS with master...
I20260808 22:33:17.811671 18232 heartbeater.cc:511] Master 127.31.87.190:40057 requested a full tablet report, sending...
I20260808 22:33:17.812893 17478 ts_manager.cc:194] Registered new tserver with Master: 9e08615db4024e949d48130f199e847a (127.31.87.131:38045)
I20260808 22:33:17.813773 17478 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.131@KRBTEST.COM'} at 127.31.87.131:55805
I20260808 22:33:17.818085 32094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:33:17.833323 17804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:33:17.822285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50158 (local address 127.31.87.190:40057)
0808 22:33:17.822416 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 22:33:17.822421 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:33:17.822824 (+ 403us) server_negotiation.cc:407] Connection header received
0808 22:33:17.822948 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:33:17.822952 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:33:17.823006 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:33:17.823088 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:33:17.823984 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:33:17.824788 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:33:17.826139 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:33:17.826489 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:33:17.826634 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:33:17.829168 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:33:17.829174 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:33:17.829177 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:33:17.829211 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:33:17.831118 (+ 1907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:33:17.831701 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:33:17.831705 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:33:17.831707 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:33:17.831806 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:33:17.832161 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:33:17.832165 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:33:17.832167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:33:17.832424 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:33:17.832537 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 22:33:17.832970 (+ 433us) server_negotiation.cc:299] Negotiation successful
0808 22:33:17.833130 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 22:33:17.842464 17478 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50158:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:33:17.848258 17478 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:33:17.880105 18259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:33:17.870156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.130:36329 (local address 127.0.0.1:48616)
0808 22:33:17.871997 (+ 1841us) negotiation.cc:107] Waiting for socket to connect
0808 22:33:17.872009 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:33:17.872093 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:33:17.872965 (+ 872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:33:17.872968 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:33:17.872987 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:33:17.873945 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:33:17.873954 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:33:17.875361 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:33:17.875365 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:33:17.878140 (+ 2775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:33:17.878150 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:33:17.879523 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:33:17.879527 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:33:17.879636 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:33:17.879680 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 22:33:17.879809 (+ 129us) client_negotiation.cc:240] Negotiation successful
0808 22:33:17.879934 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1787,"mutex_wait_us":553,"thread_start_us":95,"threads_started":1}
I20260808 22:33:17.880760 18260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:33:17.870592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48616 (local address 127.31.87.130:36329)
0808 22:33:17.872326 (+ 1734us) server_negotiation.cc:206] Beginning negotiation
0808 22:33:17.872347 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 22:33:17.872370 (+ 23us) server_negotiation.cc:407] Connection header received
0808 22:33:17.872459 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:33:17.872465 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:33:17.872652 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:33:17.872815 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:33:17.874104 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:33:17.875215 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:33:17.878306 (+ 3091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:33:17.879173 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:33:17.879287 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:33:17.879451 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 22:33:17.880498 (+ 1047us) server_negotiation.cc:299] Negotiation successful
0808 22:33:17.880605 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1624,"thread_start_us":109,"threads_started":1}
I20260808 22:33:17.882150 18255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:33:17.868974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.131:38045 (local address 127.0.0.1:43984)
0808 22:33:17.871708 (+ 2734us) negotiation.cc:107] Waiting for socket to connect
0808 22:33:17.871723 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:33:17.871818 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:33:17.872491 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:33:17.872498 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:33:17.872522 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:33:17.872649 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:33:17.872656 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:33:17.877352 (+ 4696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:33:17.877356 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:33:17.878573 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:33:17.878585 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:33:17.881719 (+ 3134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:33:17.881724 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:33:17.881819 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:33:17.881847 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 22:33:17.881902 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 22:33:17.881973 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2666,"mutex_wait_us":73,"thread_start_us":149,"threads_started":1}
I20260808 22:33:17.883925 18257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:33:17.869474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43984 (local address 127.31.87.131:38045)
0808 22:33:17.871800 (+ 2326us) server_negotiation.cc:206] Beginning negotiation
0808 22:33:17.871812 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 22:33:17.871838 (+ 26us) server_negotiation.cc:407] Connection header received
0808 22:33:17.871921 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:33:17.871926 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:33:17.872135 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:33:17.872299 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:33:17.873805 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:33:17.874999 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:33:17.878762 (+ 3763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:33:17.881564 (+ 2802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:33:17.882596 (+ 1032us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:33:17.882748 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 22:33:17.882803 (+ 55us) server_negotiation.cc:299] Negotiation successful
0808 22:33:17.882901 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2202,"thread_start_us":124,"threads_started":1}
I20260808 22:33:17.884068 17917 tablet_service.cc:1467] Processing CreateTablet for tablet 3b4ba6ce24f54749a68594c6f5febc20 (DEFAULT_TABLE table=test-table [id=1eba48a77a6a4933952aa214f636579e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:33:17.885303 17917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b4ba6ce24f54749a68594c6f5febc20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:33:17.886977 18131 tablet_service.cc:1467] Processing CreateTablet for tablet 3b4ba6ce24f54749a68594c6f5febc20 (DEFAULT_TABLE table=test-table [id=1eba48a77a6a4933952aa214f636579e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:33:17.888224 18131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b4ba6ce24f54749a68594c6f5febc20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:33:17.893726 18256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:33:17.866269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:35795 (local address 127.0.0.1:49148)
0808 22:33:17.871298 (+ 5029us) negotiation.cc:107] Waiting for socket to connect
0808 22:33:17.871350 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 22:33:17.871492 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:33:17.873541 (+ 2049us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:33:17.873545 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:33:17.873563 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:33:17.873650 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:33:17.873656 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:33:17.876381 (+ 2725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:33:17.876385 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:33:17.877675 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:33:17.877687 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:33:17.893196 (+ 15509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:33:17.893201 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:33:17.893320 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:33:17.893352 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 22:33:17.893452 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 22:33:17.893547 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4835,"mutex_wait_us":1,"thread_start_us":1357,"threads_started":1}
I20260808 22:33:17.894332 18254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:33:17.866420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49148 (local address 127.31.87.129:35795)
0808 22:33:17.872945 (+ 6525us) server_negotiation.cc:206] Beginning negotiation
0808 22:33:17.872951 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:33:17.872973 (+ 22us) server_negotiation.cc:407] Connection header received
0808 22:33:17.873048 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:33:17.873053 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:33:17.873255 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:33:17.873393 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:33:17.875140 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:33:17.876228 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:33:17.879930 (+ 3702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:33:17.880746 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:33:17.880865 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:33:17.881009 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 22:33:17.894053 (+ 13044us) server_negotiation.cc:299] Negotiation successful
0808 22:33:17.894184 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6416,"thread_start_us":139,"threads_started":1}
I20260808 22:33:17.897429 17701 tablet_service.cc:1467] Processing CreateTablet for tablet 3b4ba6ce24f54749a68594c6f5febc20 (DEFAULT_TABLE table=test-table [id=1eba48a77a6a4933952aa214f636579e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:33:17.898572 17701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b4ba6ce24f54749a68594c6f5febc20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:33:17.898910 18262 tablet_bootstrap.cc:492] T 3b4ba6ce24f54749a68594c6f5febc20 P 9e08615db4024e949d48130f199e847a: Bootstrap starting.
I20260808 22:33:17.902992 18262 tablet_bootstrap.cc:654] T 3b4ba6ce24f54749a68594c6f5febc20 P 9e08615db4024e949d48130f199e847a: Neither blocks nor log segments found. Creating new log.
I20260808 22:33:17.903861 18262 log.cc:824] T 3b4ba6ce24f54749a68594c6f5febc20 P 9e08615db4024e949d48130f199e847a: Log is configured to *not* fsync() on all Append() calls
I20260808 22:33:17.906128 18262 tablet_bootstrap.cc:492] T 3b4ba6ce24f54749a68594c6f5febc20 P 9e08615db4024e949d48130f199e847a: No bootstrap required, opened a new log
I20260808 22:33:17.906345 18262 ts_tablet_manager.cc:1397] T 3b4ba6ce24f54749a68594c6f5febc20 P 9e08615db4024e949d48130f199e847a: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.005s
I20260808 22:33:17.908852 18261 tablet_bootstrap.cc:492] T 3b4ba6ce24f54749a68594c6f5febc20 P 32dde4ef77a44e6f8171b17f62dcbcc9: Bootstrap starting.
I20260808 22:33:17.911368 18262 raft_consensus.cc:348] T 3b4ba6ce24f54749a68594c6f5febc20 P 9e08615db4024e949d48130f199e847a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "676e1dff3a2949508bb00bc94561220d" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 35795 } } peers { permanent_uuid: "32dde4ef77a44e6f8171b17f62dcbcc9" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 36329 } } peers { permanent_uuid: "9e08615db4024e949d48130f199e847a" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 38045 } }
I20260808 22:33:17.911646 18262 raft_consensus.cc:374] T 3b4ba6ce24f54749a68594c6f5febc20 P 9e08615db4024e949d48130f199e847a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:33:17.911705 18262 raft_consensus.cc:729] T 3b4ba6ce24f54749a68594c6f5febc20 P 9e08615db4024e949d48130f199e847a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e08615db4024e949d48130f199e847a, State: Initialized, Role: FOLLOWER
I20260808 22:33:17.911904 18261 tablet_bootstrap.cc:654] T 3b4ba6ce24f54749a68594c6f5febc20 P 32dde4ef77a44e6f8171b17f62dcbcc9: Neither blocks nor log segments found. Creating new log.
I20260808 22:33:17.912206 18262 consensus_queue.cc:260] T 3b4ba6ce24f54749a68594c6f5febc20 P 9e08615db4024e949d48130f199e847a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "676e1dff3a2949508bb00bc94561220d" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 35795 } } peers { permanent_uuid: "32dde4ef77a44e6f8171b17f62dcbcc9" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 36329 } } peers { permanent_uuid: "9e08615db4024e949d48130f199e847a" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 38045 } }
I20260808 22:33:17.912974 18261 log.cc:824] T 3b4ba6ce24f54749a68594c6f5febc20 P 32dde4ef77a44e6f8171b17f62dcbcc9: Log is configured to *not* fsync() on all Append() calls
I20260808 22:33:17.913439 18262 ts_tablet_manager.cc:1428] T 3b4ba6ce24f54749a68594c6f5febc20 P 9e08615db4024e949d48130f199e847a: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260808 22:33:17.913796 18232 heartbeater.cc:503] Master 127.31.87.190:40057 was elected leader, sending a full tablet report...
I20260808 22:33:17.913834 18266 tablet_bootstrap.cc:492] T 3b4ba6ce24f54749a68594c6f5febc20 P 676e1dff3a2949508bb00bc94561220d: Bootstrap starting.
I20260808 22:33:17.916038 18266 tablet_bootstrap.cc:654] T 3b4ba6ce24f54749a68594c6f5febc20 P 676e1dff3a2949508bb00bc94561220d: Neither blocks nor log segments found. Creating new log.
I20260808 22:33:17.916906 18266 log.cc:824] T 3b4ba6ce24f54749a68594c6f5febc20 P 676e1dff3a2949508bb00bc94561220d: Log is configured to *not* fsync() on all Append() calls
I20260808 22:33:17.919204 18261 tablet_bootstrap.cc:492] T 3b4ba6ce24f54749a68594c6f5febc20 P 32dde4ef77a44e6f8171b17f62dcbcc9: No bootstrap required, opened a new log
I20260808 22:33:17.919447 18261 ts_tablet_manager.cc:1397] T 3b4ba6ce24f54749a68594c6f5febc20 P 32dde4ef77a44e6f8171b17f62dcbcc9: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.003s
I20260808 22:33:17.921800 18266 tablet_bootstrap.cc:492] T 3b4ba6ce24f54749a68594c6f5febc20 P 676e1dff3a2949508bb00bc94561220d: No bootstrap required, opened a new log
I20260808 22:33:17.922000 18266 ts_tablet_manager.cc:1397] T 3b4ba6ce24f54749a68594c6f5febc20 P 676e1dff3a2949508bb00bc94561220d: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.001s
I20260808 22:33:17.924090 18261 raft_consensus.cc:348] T 3b4ba6ce24f54749a68594c6f5febc20 P 32dde4ef77a44e6f8171b17f62dcbcc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "676e1dff3a2949508bb00bc94561220d" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 35795 } } peers { permanent_uuid: "32dde4ef77a44e6f8171b17f62dcbcc9" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 36329 } } peers { permanent_uuid: "9e08615db4024e949d48130f199e847a" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 38045 } }
I20260808 22:33:17.924418 18261 raft_consensus.cc:374] T 3b4ba6ce24f54749a68594c6f5febc20 P 32dde4ef77a44e6f8171b17f62dcbcc9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:33:17.924520 18261 raft_consensus.cc:729] T 3b4ba6ce24f54749a68594c6f5febc20 P 32dde4ef77a44e6f8171b17f62dcbcc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32dde4ef77a44e6f8171b17f62dcbcc9, State: Initialized, Role: FOLLOWER
I20260808 22:33:17.925081 18261 consensus_queue.cc:260] T 3b4ba6ce24f54749a68594c6f5febc20 P 32dde4ef77a44e6f8171b17f62dcbcc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "676e1dff3a2949508bb00bc94561220d" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 35795 } } peers { permanent_uuid: "32dde4ef77a44e6f8171b17f62dcbcc9" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 36329 } } peers { permanent_uuid: "9e08615db4024e949d48130f199e847a" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 38045 } }
I20260808 22:33:17.926208 18261 ts_tablet_manager.cc:1428] T 3b4ba6ce24f54749a68594c6f5febc20 P 32dde4ef77a44e6f8171b17f62dcbcc9: Time spent starting tablet: real 0.007s user 0.003s sys 0.003s
I20260808 22:33:17.926581 18026 heartbeater.cc:503] Master 127.31.87.190:40057 was elected leader, sending a full tablet report...
I20260808 22:33:17.926595 18266 raft_consensus.cc:348] T 3b4ba6ce24f54749a68594c6f5febc20 P 676e1dff3a2949508bb00bc94561220d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "676e1dff3a2949508bb00bc94561220d" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 35795 } } peers { permanent_uuid: "32dde4ef77a44e6f8171b17f62dcbcc9" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 36329 } } peers { permanent_uuid: "9e08615db4024e949d48130f199e847a" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 38045 } }
I20260808 22:33:17.926847 18266 raft_consensus.cc:374] T 3b4ba6ce24f54749a68594c6f5febc20 P 676e1dff3a2949508bb00bc94561220d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:33:17.926898 18266 raft_consensus.cc:729] T 3b4ba6ce24f54749a68594c6f5febc20 P 676e1dff3a2949508bb00bc94561220d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 676e1dff3a2949508bb00bc94561220d, State: Initialized, Role: FOLLOWER
I20260808 22:33:17.927369 18266 consensus_queue.cc:260] T 3b4ba6ce24f54749a68594c6f5febc20 P 676e1dff3a2949508bb00bc94561220d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "676e1dff3a2949508bb00bc94561220d" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 35795 } } peers { permanent_uuid: "32dde4ef77a44e6f8171b17f62dcbcc9" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 36329 } } peers { permanent_uuid: "9e08615db4024e949d48130f199e847a" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 38045 } }
I20260808 22:33:17.928629 18266 ts_tablet_manager.cc:1428] T 3b4ba6ce24f54749a68594c6f5febc20 P 676e1dff3a2949508bb00bc94561220d: Time spent starting tablet: real 0.007s user 0.004s sys 0.003s
I20260808 22:33:17.928953 17801 heartbeater.cc:503] Master 127.31.87.190:40057 was elected leader, sending a full tablet report...
I20260808 22:33:17.983312 18268 raft_consensus.cc:482] T 3b4ba6ce24f54749a68594c6f5febc20 P 9e08615db4024e949d48130f199e847a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:33:17.983568 18268 raft_consensus.cc:504] T 3b4ba6ce24f54749a68594c6f5febc20 P 9e08615db4024e949d48130f199e847a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "676e1dff3a2949508bb00bc94561220d" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 35795 } } peers { permanent_uuid: "32dde4ef77a44e6f8171b17f62dcbcc9" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 36329 } } peers { permanent_uuid: "9e08615db4024e949d48130f199e847a" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 38045 } }
I20260808 22:33:17.985028 18268 leader_election.cc:288] T 3b4ba6ce24f54749a68594c6f5febc20 P 9e08615db4024e949d48130f199e847a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 676e1dff3a2949508bb00bc94561220d (127.31.87.129:35795), 32dde4ef77a44e6f8171b17f62dcbcc9 (127.31.87.130:36329)
I20260808 22:33:17.990538 18234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:33:17.985599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.130:36329 (local address 127.31.87.131:38387)
0808 22:33:17.986163 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0808 22:33:17.986179 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 22:33:17.986296 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:33:17.986714 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:33:17.986716 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:33:17.986737 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:33:17.986859 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:33:17.986866 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:33:17.987894 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:33:17.987897 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:33:17.989072 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:33:17.989109 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:33:17.990082 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:33:17.990086 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:33:17.990185 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:33:17.990225 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 22:33:17.990297 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 22:33:17.990369 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":297,"threads_started":1}
I20260808 22:33:17.990989 18260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:33:17.985717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:38387 (local address 127.31.87.130:36329)
0808 22:33:17.985842 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 22:33:17.985847 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:33:17.986395 (+ 548us) server_negotiation.cc:407] Connection header received
0808 22:33:17.986457 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:33:17.986461 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:33:17.986513 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:33:17.986582 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:33:17.987000 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:33:17.987753 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:33:17.989278 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:33:17.989851 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:33:17.989964 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:33:17.990019 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 22:33:17.990817 (+ 798us) server_negotiation.cc:299] Negotiation successful
0808 22:33:17.990900 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 22:33:17.991391 18254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:33:17.985380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:39269 (local address 127.31.87.129:35795)
0808 22:33:17.985505 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 22:33:17.985510 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:33:17.986972 (+ 1462us) server_negotiation.cc:407] Connection header received
0808 22:33:17.987191 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:33:17.987195 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:33:17.987249 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:33:17.987298 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:33:17.987716 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:33:17.988455 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:33:17.989971 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:33:17.990578 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:33:17.990683 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:33:17.990736 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 22:33:17.991177 (+ 441us) server_negotiation.cc:299] Negotiation successful
0808 22:33:17.991262 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 22:33:17.991554 17959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b4ba6ce24f54749a68594c6f5febc20" candidate_uuid: "9e08615db4024e949d48130f199e847a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32dde4ef77a44e6f8171b17f62dcbcc9" is_pre_election: true
I20260808 22:33:17.991775 18274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:33:17.985322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:35795 (local address 127.31.87.131:39269)
0808 22:33:17.986868 (+ 1546us) negotiation.cc:107] Waiting for socket to connect
0808 22:33:17.986885 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 22:33:17.987047 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:33:17.987436 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:33:17.987439 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:33:17.987456 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:33:17.987579 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:33:17.987586 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:33:17.988613 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:33:17.988617 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:33:17.989806 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:33:17.989820 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:33:17.990898 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:33:17.990902 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:33:17.990998 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:33:17.991025 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 22:33:17.991550 (+ 525us) client_negotiation.cc:240] Negotiation successful
0808 22:33:17.991632 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":744}
I20260808 22:33:17.991879 17959 raft_consensus.cc:2454] T 3b4ba6ce24f54749a68594c6f5febc20 P 32dde4ef77a44e6f8171b17f62dcbcc9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e08615db4024e949d48130f199e847a in term 0.
I20260808 22:33:17.992370 17725 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b4ba6ce24f54749a68594c6f5febc20" candidate_uuid: "9e08615db4024e949d48130f199e847a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "676e1dff3a2949508bb00bc94561220d" is_pre_election: true
I20260808 22:33:17.992635 17725 raft_consensus.cc:2454] T 3b4ba6ce24f54749a68594c6f5febc20 P 676e1dff3a2949508bb00bc94561220d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e08615db4024e949d48130f199e847a in term 0.
I20260808 22:33:17.993245 18060 leader_election.cc:302] T 3b4ba6ce24f54749a68594c6f5febc20 P 9e08615db4024e949d48130f199e847a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 676e1dff3a2949508bb00bc94561220d, 9e08615db4024e949d48130f199e847a; no voters:
I20260808 22:33:17.993767 18268 raft_consensus.cc:2781] T 3b4ba6ce24f54749a68594c6f5febc20 P 9e08615db4024e949d48130f199e847a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:33:17.993840 18268 raft_consensus.cc:482] T 3b4ba6ce24f54749a68594c6f5febc20 P 9e08615db4024e949d48130f199e847a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:33:17.993875 18268 raft_consensus.cc:3037] T 3b4ba6ce24f54749a68594c6f5febc20 P 9e08615db4024e949d48130f199e847a [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:33:17.995234 18268 raft_consensus.cc:504] T 3b4ba6ce24f54749a68594c6f5febc20 P 9e08615db4024e949d48130f199e847a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "676e1dff3a2949508bb00bc94561220d" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 35795 } } peers { permanent_uuid: "32dde4ef77a44e6f8171b17f62dcbcc9" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 36329 } } peers { permanent_uuid: "9e08615db4024e949d48130f199e847a" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 38045 } }
I20260808 22:33:17.995584 18268 leader_election.cc:288] T 3b4ba6ce24f54749a68594c6f5febc20 P 9e08615db4024e949d48130f199e847a [CANDIDATE]: Term 1 election: Requested vote from peers 676e1dff3a2949508bb00bc94561220d (127.31.87.129:35795), 32dde4ef77a44e6f8171b17f62dcbcc9 (127.31.87.130:36329)
I20260808 22:33:17.996466 17959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b4ba6ce24f54749a68594c6f5febc20" candidate_uuid: "9e08615db4024e949d48130f199e847a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32dde4ef77a44e6f8171b17f62dcbcc9"
I20260808 22:33:17.996587 17959 raft_consensus.cc:3037] T 3b4ba6ce24f54749a68594c6f5febc20 P 32dde4ef77a44e6f8171b17f62dcbcc9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:33:17.997918 17959 raft_consensus.cc:2454] T 3b4ba6ce24f54749a68594c6f5febc20 P 32dde4ef77a44e6f8171b17f62dcbcc9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e08615db4024e949d48130f199e847a in term 1.
I20260808 22:33:17.998371 17725 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b4ba6ce24f54749a68594c6f5febc20" candidate_uuid: "9e08615db4024e949d48130f199e847a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "676e1dff3a2949508bb00bc94561220d"
I20260808 22:33:17.998523 17725 raft_consensus.cc:3037] T 3b4ba6ce24f54749a68594c6f5febc20 P 676e1dff3a2949508bb00bc94561220d [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:33:17.999833 17725 raft_consensus.cc:2454] T 3b4ba6ce24f54749a68594c6f5febc20 P 676e1dff3a2949508bb00bc94561220d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e08615db4024e949d48130f199e847a in term 1.
I20260808 22:33:18.000290 18060 leader_election.cc:302] T 3b4ba6ce24f54749a68594c6f5febc20 P 9e08615db4024e949d48130f199e847a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 676e1dff3a2949508bb00bc94561220d, 9e08615db4024e949d48130f199e847a; no voters:
I20260808 22:33:18.000618 18268 raft_consensus.cc:2781] T 3b4ba6ce24f54749a68594c6f5febc20 P 9e08615db4024e949d48130f199e847a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:33:18.000840 18268 raft_consensus.cc:686] T 3b4ba6ce24f54749a68594c6f5febc20 P 9e08615db4024e949d48130f199e847a [term 1 LEADER]: Becoming Leader. State: Replica: 9e08615db4024e949d48130f199e847a, State: Running, Role: LEADER
I20260808 22:33:18.001282 18268 consensus_queue.cc:237] T 3b4ba6ce24f54749a68594c6f5febc20 P 9e08615db4024e949d48130f199e847a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "676e1dff3a2949508bb00bc94561220d" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 35795 } } peers { permanent_uuid: "32dde4ef77a44e6f8171b17f62dcbcc9" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 36329 } } peers { permanent_uuid: "9e08615db4024e949d48130f199e847a" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 38045 } }
Aug 08 22:33:17 dist-test-slave-266d krb5kdc[16747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228397, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for HTTP/127.31.87.148@KRBTEST.COM
I20260808 22:33:18.005581 17477 catalog_manager.cc:5495] T 3b4ba6ce24f54749a68594c6f5febc20 P 9e08615db4024e949d48130f199e847a reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e08615db4024e949d48130f199e847a (127.31.87.131). New cstate: current_term: 1 leader_uuid: "9e08615db4024e949d48130f199e847a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "676e1dff3a2949508bb00bc94561220d" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 35795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32dde4ef77a44e6f8171b17f62dcbcc9" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 36329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e08615db4024e949d48130f199e847a" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 38045 } health_report { overall_health: HEALTHY } } }
W20260808 22:33:18.039918 17802 tablet.cc:2369] T 3b4ba6ce24f54749a68594c6f5febc20 P 676e1dff3a2949508bb00bc94561220d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 22:33:18.047736 18233 tablet.cc:2369] T 3b4ba6ce24f54749a68594c6f5febc20 P 9e08615db4024e949d48130f199e847a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:33:18.055835 18257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:33:18.049859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43998 (local address 127.31.87.131:38045)
0808 22:33:18.049985 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 22:33:18.049989 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:33:18.050003 (+ 14us) server_negotiation.cc:407] Connection header received
0808 22:33:18.050083 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:33:18.050086 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:33:18.050148 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:33:18.050248 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:33:18.050633 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:33:18.051440 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:33:18.052930 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:33:18.053303 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:33:18.053399 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:33:18.054248 (+ 849us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:33:18.054761 (+ 513us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:33:18.054880 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 22:33:18.055592 (+ 712us) server_negotiation.cc:299] Negotiation successful
0808 22:33:18.055659 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 22:33:18.063572 17959 raft_consensus.cc:1261] T 3b4ba6ce24f54749a68594c6f5febc20 P 32dde4ef77a44e6f8171b17f62dcbcc9 [term 1 FOLLOWER]: Refusing update from remote peer 9e08615db4024e949d48130f199e847a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:33:18.064314 18276 consensus_queue.cc:1035] T 3b4ba6ce24f54749a68594c6f5febc20 P 9e08615db4024e949d48130f199e847a [LEADER]: Connected to new peer: Peer: permanent_uuid: "32dde4ef77a44e6f8171b17f62dcbcc9" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 36329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:33:18.065625 17725 raft_consensus.cc:1261] T 3b4ba6ce24f54749a68594c6f5febc20 P 676e1dff3a2949508bb00bc94561220d [term 1 FOLLOWER]: Refusing update from remote peer 9e08615db4024e949d48130f199e847a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:33:18.066215 18276 consensus_queue.cc:1035] T 3b4ba6ce24f54749a68594c6f5febc20 P 9e08615db4024e949d48130f199e847a [LEADER]: Connected to new peer: Peer: permanent_uuid: "676e1dff3a2949508bb00bc94561220d" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 35795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:33:18.083691 18283 mvcc.cc:205] Tried to move back new op lower bound from 7316391518458908672 to 7316391518216781824. Current Snapshot: MvccSnapshot[applied={T|T < 7316391518458908672}]
I20260808 22:33:18.087854 18280 mvcc.cc:205] Tried to move back new op lower bound from 7316391518458908672 to 7316391518216781824. Current Snapshot: MvccSnapshot[applied={T|T < 7316391518458908672 or (T in {7316391518458908672})}]
I20260808 22:33:18.094278 17478 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:33:18.094455 17478 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:33:18.097018 17478 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 936307bb66fd4309a8a3c78fe120d1c4: Sending DeleteTablet for 3 replicas of tablet 3b4ba6ce24f54749a68594c6f5febc20
I20260808 22:33:18.098491 17701 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b4ba6ce24f54749a68594c6f5febc20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:33:18 UTC) from {username='kudu', principal='kudu/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:49148
I20260808 22:33:18.099107 17917 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b4ba6ce24f54749a68594c6f5febc20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:33:18 UTC) from {username='kudu', principal='kudu/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:48616
I20260808 22:33:18.099618 18131 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b4ba6ce24f54749a68594c6f5febc20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:33:18 UTC) from {username='kudu', principal='kudu/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:43984
I20260808 22:33:18.100835 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 17601
I20260808 22:33:18.104346 18293 tablet_replica.cc:330] T 3b4ba6ce24f54749a68594c6f5febc20 P 32dde4ef77a44e6f8171b17f62dcbcc9: stopping tablet replica
I20260808 22:33:18.104650 18293 raft_consensus.cc:2229] T 3b4ba6ce24f54749a68594c6f5febc20 P 32dde4ef77a44e6f8171b17f62dcbcc9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:33:18.104961 18293 raft_consensus.cc:2258] T 3b4ba6ce24f54749a68594c6f5febc20 P 32dde4ef77a44e6f8171b17f62dcbcc9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:33:18.105353 18296 tablet_replica.cc:330] T 3b4ba6ce24f54749a68594c6f5febc20 P 9e08615db4024e949d48130f199e847a: stopping tablet replica
I20260808 22:33:18.105680 18296 raft_consensus.cc:2229] T 3b4ba6ce24f54749a68594c6f5febc20 P 9e08615db4024e949d48130f199e847a [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:33:18.106089 18296 raft_consensus.cc:2258] T 3b4ba6ce24f54749a68594c6f5febc20 P 9e08615db4024e949d48130f199e847a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:33:18.108377 18293 ts_tablet_manager.cc:1905] T 3b4ba6ce24f54749a68594c6f5febc20 P 32dde4ef77a44e6f8171b17f62dcbcc9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:33:18.111984 18293 ts_tablet_manager.cc:1918] T 3b4ba6ce24f54749a68594c6f5febc20 P 32dde4ef77a44e6f8171b17f62dcbcc9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:33:18.112082 18293 log.cc:1196] T 3b4ba6ce24f54749a68594c6f5febc20 P 32dde4ef77a44e6f8171b17f62dcbcc9: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ts-1/wal/wals/3b4ba6ce24f54749a68594c6f5febc20
I20260808 22:33:18.112391 18293 ts_tablet_manager.cc:1939] T 3b4ba6ce24f54749a68594c6f5febc20 P 32dde4ef77a44e6f8171b17f62dcbcc9: Deleting consensus metadata
I20260808 22:33:18.113700 17454 catalog_manager.cc:4841] TS 32dde4ef77a44e6f8171b17f62dcbcc9 (127.31.87.130:36329): tablet 3b4ba6ce24f54749a68594c6f5febc20 (table test-table [id=1eba48a77a6a4933952aa214f636579e]) successfully deleted
I20260808 22:33:18.114902 18296 ts_tablet_manager.cc:1905] T 3b4ba6ce24f54749a68594c6f5febc20 P 9e08615db4024e949d48130f199e847a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:33:18.117666 18296 ts_tablet_manager.cc:1918] T 3b4ba6ce24f54749a68594c6f5febc20 P 9e08615db4024e949d48130f199e847a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:33:18.117761 18296 log.cc:1196] T 3b4ba6ce24f54749a68594c6f5febc20 P 9e08615db4024e949d48130f199e847a: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786228291013016-32094-0/minicluster-data/ts-2/wal/wals/3b4ba6ce24f54749a68594c6f5febc20
I20260808 22:33:18.118100 18296 ts_tablet_manager.cc:1939] T 3b4ba6ce24f54749a68594c6f5febc20 P 9e08615db4024e949d48130f199e847a: Deleting consensus metadata
I20260808 22:33:18.119257 17452 catalog_manager.cc:4841] TS 9e08615db4024e949d48130f199e847a (127.31.87.131:38045): tablet 3b4ba6ce24f54749a68594c6f5febc20 (table test-table [id=1eba48a77a6a4933952aa214f636579e]) successfully deleted
W20260808 22:33:18.123740 17454 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.87.129:35795 (error 108)
I20260808 22:33:18.124693 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 17811
I20260808 22:33:18.142452 18256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:33:18.142009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:35795 (local address 127.0.0.1:49154)
0808 22:33:18.142254 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0808 22:33:18.142342 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.129:35795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152}
W20260808 22:33:18.142809 17454 catalog_manager.cc:4584] TS 676e1dff3a2949508bb00bc94561220d (127.31.87.129:35795): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3b4ba6ce24f54749a68594c6f5febc20: Network error: Client connection negotiation failed: client connection to 127.31.87.129:35795: connect: Connection refused (error 111)
I20260808 22:33:18.153398 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 18035
I20260808 22:33:18.192929 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 17427
I20260808 22:33:18.206758 32094 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 22:33:18.550328 32094 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 22:33:18.550472 32094 mini_ranger.cc:66] Stopping Ranger...
I20260808 22:33:18.894457 32094 mini_ranger.cc:68] Stopped Ranger
2026-08-08 22:33:18.894 UTC [16800] LOG: received smart shutdown request
2026-08-08 22:33:18.898 UTC [16800] LOG: background worker "logical replication launcher" (PID 16815) exited with exit code 1
2026-08-08 22:33:18.898 UTC [16806] LOG: shutting down
2026-08-08 22:33:19.036 UTC [16800] LOG: database system is shut down
2026-08-08T22:33:19Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (44936 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:33:19 dist-test-slave-266d krb5kdc[18310](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:33:19 dist-test-slave-266d krb5kdc[18310](info): set up 2 sockets
Aug 08 22:33:19 dist-test-slave-266d krb5kdc[18310](info): commencing operation
krb5kdc: starting...
W20260808 22:33:20.302673 32094 mini_kdc.cc:121] Time spent starting KDC: real 1.130s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Aug 08 22:33:20 dist-test-slave-266d krb5kdc[18310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228400, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:33:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:33:20Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.31.87.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.31.87.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.31.87.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/rangeradmin_127.31.87.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.31.87.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/rangeradmin_127.31.87.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.31.87.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.31.87.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.31.87.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/rangerlookup_127.31.87.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.31.87.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/rangerlookup_127.31.87.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.31.87.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.31.87.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.31.87.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/HTTP_127.31.87.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.31.87.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/HTTP_127.31.87.148@KRBTEST.COM.keytab.
I20260808 22:33:20.456689 32094 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskOEAG_G/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/postgres -l logfile start
2026-08-08 22:33:22.677 UTC [18355] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 22:33:22.677 UTC [18355] LOG: listening on IPv4 address "127.31.87.148", port 44109
2026-08-08 22:33:22.681 UTC [18355] LOG: listening on Unix socket "/tmp/.s.PGSQL.44109"
2026-08-08 22:33:22.701 UTC [18359] LOG: database system was shut down at 2026-08-08 22:33:21 UTC
2026-08-08 22:33:22.704 UTC [18355] LOG: database system is ready to accept connections
I20260808 22:33:23.755864 32094 mini_postgres.cc:96] Postgres bound to 44109
2026-08-08 22:33:23.762 UTC [18371] FATAL: database "slave" does not exist
127.31.87.148:44109 - accepting connections
I20260808 22:33:23.763345 32094 mini_ranger.cc:161] Starting Ranger...
I20260808 22:33:23.777949 32094 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 22:33:24.190551 32094 mini_ranger.cc:87] Created ranger Postgres database
I20260808 22:33:24.190644 32094 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ranger-admin
2026-08-08 22:33:24,449 [I] DB FLAVOR :POSTGRES
2026-08-08 22:33:24,449 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 22:33:24,449 [I] Checking connection..
2026-08-08 22:33:24,449 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOEAG_G/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOEAG_G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.87.148:44109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 22:33:24,700 [I] Checking connection passed.
2026-08-08 22:33:24,700 [I] --------- Verifying version history table ---------
2026-08-08 22:33:24,701 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOEAG_G/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOEAG_G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.87.148:44109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 22:33:24,955 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 22:33:24,955 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOEAG_G/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOEAG_G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.87.148:44109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 22:33:25,192 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 22:33:25,193 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 22:33:25,193 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOEAG_G/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOEAG_G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.87.148:44109/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskOEAG_G/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 22:33:25.436 UTC [18459] WARNING: there is no transaction in progress
2026-08-08 22:33:25,445 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 22:33:25,446 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOEAG_G/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOEAG_G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.87.148:44109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 22:33:25,702 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 22:33:25,702 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 22:33:25,702 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOEAG_G/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOEAG_G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.87.148:44109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 22:33:25,940 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOEAG_G/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOEAG_G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.87.148:44109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 22:33:26,191 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOEAG_G/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOEAG_G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.87.148:44109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-266d.c.gcp-upstream.internal','N') ;"
2026-08-08 22:33:26,438 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOEAG_G/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOEAG_G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.87.148:44109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 22:33:26,695 [I] Table x_portal_user does not exist in database ranger
2026-08-08 22:33:26,695 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOEAG_G/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOEAG_G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.87.148:44109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 22:33:26,948 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 22:33:26,949 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOEAG_G/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOEAG_G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.87.148:44109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 22:33:27,193 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOEAG_G/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOEAG_G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.87.148:44109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 22:33:27,445 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 22:33:27,446 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOEAG_G/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOEAG_G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.87.148:44109/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskOEAG_G/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 22:33:28.334 UTC [18624] WARNING: there is no transaction in progress
2026-08-08 22:33:28.348 UTC [18624] WARNING: there is no transaction in progress
2026-08-08 22:33:28.362 UTC [18624] WARNING: there is no transaction in progress
2026-08-08 22:33:28.375 UTC [18624] WARNING: there is no transaction in progress
2026-08-08 22:33:28.389 UTC [18624] WARNING: there is no transaction in progress
2026-08-08 22:33:28.404 UTC [18624] WARNING: there is no transaction in progress
2026-08-08 22:33:28.452 UTC [18624] WARNING: there is no transaction in progress
2026-08-08 22:33:28.457 UTC [18624] WARNING: there is no transaction in progress
2026-08-08 22:33:28.469 UTC [18624] WARNING: there is no transaction in progress
2026-08-08 22:33:28.477 UTC [18624] WARNING: there is no transaction in progress
2026-08-08 22:33:28.487 UTC [18624] WARNING: there is no transaction in progress
2026-08-08 22:33:28.498 UTC [18624] WARNING: there is no transaction in progress
2026-08-08 22:33:28.507 UTC [18624] WARNING: there is no transaction in progress
2026-08-08 22:33:28.512 UTC [18624] WARNING: there is no transaction in progress
2026-08-08 22:33:28.518 UTC [18624] WARNING: there is no transaction in progress
2026-08-08 22:33:28.523 UTC [18624] WARNING: there is no transaction in progress
2026-08-08 22:33:29,066 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 22:33:29,066 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOEAG_G/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOEAG_G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.87.148:44109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 22:33:29,310 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 22:33:29,311 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOEAG_G/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOEAG_G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.87.148:44109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 22:33:29,554 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 22:33:29,554 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOEAG_G/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOEAG_G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.87.148:44109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 22:33:29,776 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 22:33:29,776 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOEAG_G/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOEAG_G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.87.148:44109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 22:33:30,002 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOEAG_G/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOEAG_G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.87.148:44109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 22:33:30,229 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOEAG_G/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOEAG_G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.87.148:44109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-266d.c.gcp-upstream.internal';"
2026-08-08 22:33:30,476 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 22:33:30,476 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOEAG_G/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOEAG_G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.87.148:44109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 22:33:30,733 [I] DB_PATCHES have already been applied
I20260808 22:33:30.738420 32094 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ranger-admin:/tmp/dist-test-taskOEAG_G/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskOEAG_G/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskOEAG_G/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOEAG_G/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 22:33:30.738493 32094 mini_ranger.cc:193] Using host: 127.31.87.148
I20260808 22:33:30.742533 32094 mini_ranger.cc:227] Ranger admin URL: http://127.31.87.148:59949
Aug 08, 2026 10:33:31 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 10:33:31 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 10:33:31 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 10:33:31 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 10:33:31 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 10:33:31 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 10:33:31 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 22:33:31.975160 32094 mini_ranger.cc:160] Time spent starting Ranger: real 8.212s user 0.000s sys 0.010s
Aug 08, 2026 10:33:51 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 10:33:51 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 08, 2026 10:33:51 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 10:33:51 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 10:33:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 10:33:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 10:33:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 10:33:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 10:33:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 10:33:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 10:33:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 10:33:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 10:33:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 10:33:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 10:33:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 10:33:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 10:33:51 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 10:33:52 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 22:33:53.372277 32094 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.31.87.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.31.87.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.31.87.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/rangerkms_127.31.87.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.31.87.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/rangerkms_127.31.87.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.31.87.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.31.87.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.31.87.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/HTTP_127.31.87.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.31.87.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/HTTP_127.31.87.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 22:33:53 dist-test-slave-266d krb5kdc[18310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228433, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 22:33:53.469873 32094 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 22:33:53.480077 32094 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 22:33:54.274302 32094 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 22:33:54.274396 32094 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ranger-kms
I20260808 22:33:54.274415 32094 mini_ranger_kms.cc:227] Using postgres at 127.31.87.148:44109
2026-08-08 22:33:54,443 [I] DB FLAVOR :POSTGRES
2026-08-08 22:33:54,443 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 22:33:54,443 [I] Checking connection
2026-08-08 22:33:54,443 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOEAG_G/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.87.148:44109/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 22:33:54,694 [I] connection success
2026-08-08 22:33:54,694 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 22:33:54,694 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 22:33:54,694 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOEAG_G/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.87.148:44109/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 22:33:54,966 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 22:33:54,966 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 22:33:54,967 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 22:33:54,967 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOEAG_G/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.87.148:44109/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 22:33:55,235 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 22:33:55.639803 32094 mini_ranger_kms.cc:310] Created kms service
I20260808 22:33:55.847595 32094 mini_ranger_kms.cc:326] Created kudu user
I20260808 22:33:55.890934 32094 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 22:33:55 dist-test-slave-266d krb5kdc[18310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228433, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.31.87.148@KRBTEST.COM
I20260808 22:33:56.257417 32094 mini_ranger_kms.cc:384] Added ranger policy
I20260808 22:33:56.257640 32094 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ranger-kms:/tmp/dist-test-taskOEAG_G/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOEAG_G/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskOEAG_G/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 22:33:56.257663 32094 mini_ranger_kms.cc:240] Using host: 127.31.87.148
I20260808 22:33:56.262993 32094 mini_ranger_kms.cc:276] Ranger KMS PID: 18888
I20260808 22:33:56.263064 32094 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.31.87.148:41391
Aug 08, 2026 10:33:57 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 10:33:57 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 10:33:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 10:33:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 10:33:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 10:33:57 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 22:33:58 dist-test-slave-266d krb5kdc[18310](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786228438, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.87.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:33:58.664778 32094 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.195s user 0.014s sys 0.140s
I20260808 22:33:58.664885 32094 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 22:33:58.664956 32094 mini_ranger_kms.cc:401] 127.31.87.148:41391/kms/v1/keys
Aug 08 22:33:58 dist-test-slave-266d krb5kdc[18310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228438, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.87.148@KRBTEST.COM for HTTP/127.31.87.148@KRBTEST.COM
Aug 08 22:33:59 dist-test-slave-266d krb5kdc[18310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228438, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.87.148@KRBTEST.COM for HTTP/127.31.87.148@KRBTEST.COM
Aug 08 22:34:01 dist-test-slave-266d krb5kdc[18310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228441, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.31.87.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
I20260808 22:34:01.447687 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:45947
--webserver_interface=127.31.87.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:38239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:45947
--ranger_config_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.87.148:41391/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:34:01.576187 18961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:34:01.576442 18961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:34:01.576483 18961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:34:01.580432 18961 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:34:01.580490 18961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:34:01.580511 18961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:34:01.580528 18961 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:34:01.580550 18961 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:34:01.585810 18961 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:38239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/master-0/wal
--ranger_kms_url=127.31.87.148:41391/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:45947
--ranger_config_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:45947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.87.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18961
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:34:01.586769 18961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:34:01.587720 18961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:34:01.596306 18970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:34:01.596273 18966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:34:01.596580 18961 server_base.cc:1034] running on GCE node
W20260808 22:34:01.596660 18967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:34:01.597378 18961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:34:01.598666 18961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:34:01.599853 18961 hybrid_clock.cc:648] HybridClock initialized: now 1786228441599831 us; error 34 us; skew 500 ppm
Aug 08 22:34:01 dist-test-slave-266d krb5kdc[18310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228441, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:34:01.603150 18961 init.cc:377] Logged in from keytab as kudu/127.31.87.190@KRBTEST.COM (short username kudu)
I20260808 22:34:01.604233 18961 webserver.cc:460] Webserver started at http://127.31.87.190:38249/ using document root <none> and password file <none>
I20260808 22:34:01.604867 18961 fs_manager.cc:359] Metadata directory not provided
I20260808 22:34:01.604945 18961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:34:01.605211 18961 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 22:34:01 dist-test-slave-266d krb5kdc[18310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228441, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for HTTP/127.31.87.148@KRBTEST.COM
I20260808 22:34:01.664492 18961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/master-0/data/instance:
uuid: "ddb477d3689f4bbb812a0b93b43c1673"
format_stamp: "Formatted at 2026-08-08 22:34:01 on dist-test-slave-266d"
server_key: "dc3c88f7156b2423d3f45a9212bc30ad"
server_key_iv: "aabd6a2104c0c08ee7e0f42b11a4cb7a"
server_key_version: "kuduclusterkey@0"
I20260808 22:34:01.665397 18961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/master-0/wal/instance:
uuid: "ddb477d3689f4bbb812a0b93b43c1673"
format_stamp: "Formatted at 2026-08-08 22:34:01 on dist-test-slave-266d"
server_key: "dc3c88f7156b2423d3f45a9212bc30ad"
server_key_iv: "aabd6a2104c0c08ee7e0f42b11a4cb7a"
server_key_version: "kuduclusterkey@0"
I20260808 22:34:01.670068 18961 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 22:34:01.687475 18984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:01.689152 18961 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 22:34:01.689318 18961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "ddb477d3689f4bbb812a0b93b43c1673"
format_stamp: "Formatted at 2026-08-08 22:34:01 on dist-test-slave-266d"
server_key: "dc3c88f7156b2423d3f45a9212bc30ad"
server_key_iv: "aabd6a2104c0c08ee7e0f42b11a4cb7a"
server_key_version: "kuduclusterkey@0"
I20260808 22:34:01.689468 18961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:34:01.718411 18961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:34:01.720960 18961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:34:01.721144 18961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:34:01.727973 18961 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.190:45947
I20260808 22:34:01.727998 19060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.190:45947 every 8 connection(s)
I20260808 22:34:01.728900 18961 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:34:01.731559 19061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:34:01.735428 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 18961
I20260808 22:34:01.735531 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/master-0/wal/instance
I20260808 22:34:01.735777 32094 external_mini_cluster.cc:1428] Setting key f616a2dd3f410e09f9de70b838961a87
I20260808 22:34:01.737082 19061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddb477d3689f4bbb812a0b93b43c1673: Bootstrap starting.
I20260808 22:34:01.739470 19061 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ddb477d3689f4bbb812a0b93b43c1673: Neither blocks nor log segments found. Creating new log.
I20260808 22:34:01.740165 19061 log.cc:824] T 00000000000000000000000000000000 P ddb477d3689f4bbb812a0b93b43c1673: Log is configured to *not* fsync() on all Append() calls
I20260808 22:34:01.741901 19061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddb477d3689f4bbb812a0b93b43c1673: No bootstrap required, opened a new log
Aug 08 22:34:01 dist-test-slave-266d krb5kdc[18310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228441, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:34:01.745288 19061 raft_consensus.cc:348] T 00000000000000000000000000000000 P ddb477d3689f4bbb812a0b93b43c1673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddb477d3689f4bbb812a0b93b43c1673" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45947 } }
I20260808 22:34:01.745494 19061 raft_consensus.cc:374] T 00000000000000000000000000000000 P ddb477d3689f4bbb812a0b93b43c1673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:34:01.745551 19061 raft_consensus.cc:729] T 00000000000000000000000000000000 P ddb477d3689f4bbb812a0b93b43c1673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddb477d3689f4bbb812a0b93b43c1673, State: Initialized, Role: FOLLOWER
I20260808 22:34:01.745973 19061 consensus_queue.cc:260] T 00000000000000000000000000000000 P ddb477d3689f4bbb812a0b93b43c1673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddb477d3689f4bbb812a0b93b43c1673" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45947 } }
I20260808 22:34:01.746122 19061 raft_consensus.cc:388] T 00000000000000000000000000000000 P ddb477d3689f4bbb812a0b93b43c1673 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:34:01.746194 19061 raft_consensus.cc:482] T 00000000000000000000000000000000 P ddb477d3689f4bbb812a0b93b43c1673 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:34:01.746297 19061 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ddb477d3689f4bbb812a0b93b43c1673 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:34:01.747255 19061 raft_consensus.cc:504] T 00000000000000000000000000000000 P ddb477d3689f4bbb812a0b93b43c1673 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddb477d3689f4bbb812a0b93b43c1673" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45947 } }
I20260808 22:34:01.747599 19061 leader_election.cc:302] T 00000000000000000000000000000000 P ddb477d3689f4bbb812a0b93b43c1673 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ddb477d3689f4bbb812a0b93b43c1673; no voters:
I20260808 22:34:01.747920 19061 leader_election.cc:288] T 00000000000000000000000000000000 P ddb477d3689f4bbb812a0b93b43c1673 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:34:01.748003 19066 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ddb477d3689f4bbb812a0b93b43c1673 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:34:01.748193 19066 raft_consensus.cc:686] T 00000000000000000000000000000000 P ddb477d3689f4bbb812a0b93b43c1673 [term 1 LEADER]: Becoming Leader. State: Replica: ddb477d3689f4bbb812a0b93b43c1673, State: Running, Role: LEADER
I20260808 22:34:01.748478 19066 consensus_queue.cc:237] T 00000000000000000000000000000000 P ddb477d3689f4bbb812a0b93b43c1673 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddb477d3689f4bbb812a0b93b43c1673" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45947 } }
I20260808 22:34:01.748644 19064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:01.736989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44942 (local address 127.31.87.190:45947)
0808 22:34:01.737377 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:01.737393 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:01.737426 (+ 33us) server_negotiation.cc:407] Connection header received
0808 22:34:01.738028 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:01.738043 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:01.738338 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:01.738594 (+ 256us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:34:01.739478 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:01.740454 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:01.741312 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:01.741752 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:01.741907 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:01.744163 (+ 2256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:34:01.744169 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:34:01.744181 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:34:01.744207 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:34:01.745790 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:01.746217 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:01.746223 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:01.746229 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:01.746365 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:01.746630 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:01.746634 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:01.746635 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:01.747122 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:34:01.747311 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:01.747500 (+ 189us) server_negotiation.cc:299] Negotiation successful
0808 22:34:01.747767 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":150,"threads_started":1}
I20260808 22:34:01.749064 19061 sys_catalog.cc:564] T 00000000000000000000000000000000 P ddb477d3689f4bbb812a0b93b43c1673 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:34:01.750519 19067 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddb477d3689f4bbb812a0b93b43c1673 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ddb477d3689f4bbb812a0b93b43c1673" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddb477d3689f4bbb812a0b93b43c1673" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45947 } } }
I20260808 22:34:01.750638 19067 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddb477d3689f4bbb812a0b93b43c1673 [sys.catalog]: This master's current role is: LEADER
I20260808 22:34:01.750982 19082 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:34:01.751212 19061 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 22:34:01.751915 19069 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddb477d3689f4bbb812a0b93b43c1673 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ddb477d3689f4bbb812a0b93b43c1673. Latest consensus state: current_term: 1 leader_uuid: "ddb477d3689f4bbb812a0b93b43c1673" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddb477d3689f4bbb812a0b93b43c1673" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45947 } } }
I20260808 22:34:01.752007 19069 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddb477d3689f4bbb812a0b93b43c1673 [sys.catalog]: This master's current role is: LEADER
I20260808 22:34:01.756564 19082 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:34:01.762740 19082 catalog_manager.cc:1347] Generated new cluster ID: 2116a155a15b44bcacf0a9a9f47fff39
I20260808 22:34:01.762812 19082 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:34:01.768534 19082 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:34:01.769675 19082 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:34:01.774483 19082 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ddb477d3689f4bbb812a0b93b43c1673: Generated new TSK 0
I20260808 22:34:01.775058 19082 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.87.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab.
Entry for principal kudu/127.31.87.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab.
I20260808 22:34:02.480544 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.87.148:41391/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.129:0
--local_ip_for_outbound_sockets=127.31.87.129
--webserver_interface=127.31.87.129
--webserver_port=0
--tserver_master_addrs=127.31.87.190:45947
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab
--principal=kudu/127.31.87.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:38239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:34:02.627895 19132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:34:02.628124 19132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:34:02.628181 19132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:34:02.632108 19132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:34:02.632157 19132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:34:02.632230 19132 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.129
I20260808 22:34:02.637413 19132 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:38239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.31.87.148:41391/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab
--principal=kudu/127.31.87.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.87.129
--webserver_port=0
--tserver_master_addrs=127.31.87.190:45947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19132
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.129
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:34:02.638301 19132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:34:02.639210 19132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:34:02.650727 19139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:34:02.650734 19140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:34:02.651077 19132 server_base.cc:1034] running on GCE node
W20260808 22:34:02.651077 19145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:34:02.652276 19132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:34:02.653045 19132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:34:02.654320 19132 hybrid_clock.cc:648] HybridClock initialized: now 1786228442654289 us; error 41 us; skew 500 ppm
Aug 08 22:34:02 dist-test-slave-266d krb5kdc[18310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228442, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:34:02.657557 19132 init.cc:377] Logged in from keytab as kudu/127.31.87.129@KRBTEST.COM (short username kudu)
I20260808 22:34:02.658397 19132 webserver.cc:460] Webserver started at http://127.31.87.129:46065/ using document root <none> and password file <none>
I20260808 22:34:02.658903 19132 fs_manager.cc:359] Metadata directory not provided
I20260808 22:34:02.658967 19132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:34:02.659178 19132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 22:34:02 dist-test-slave-266d krb5kdc[18310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228442, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.129@KRBTEST.COM for HTTP/127.31.87.148@KRBTEST.COM
I20260808 22:34:02.672148 19132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-0/data/instance:
uuid: "4750e9a921794581ae20ed0f1e6d7b4a"
format_stamp: "Formatted at 2026-08-08 22:34:02 on dist-test-slave-266d"
server_key: "8f4aa29cd9b03f0c8c26f3f4d4826658"
server_key_iv: "f02391f089d936fa398fa6b2e51bac47"
server_key_version: "kuduclusterkey@0"
I20260808 22:34:02.672808 19132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-0/wal/instance:
uuid: "4750e9a921794581ae20ed0f1e6d7b4a"
format_stamp: "Formatted at 2026-08-08 22:34:02 on dist-test-slave-266d"
server_key: "8f4aa29cd9b03f0c8c26f3f4d4826658"
server_key_iv: "f02391f089d936fa398fa6b2e51bac47"
server_key_version: "kuduclusterkey@0"
I20260808 22:34:02.677922 19132 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 22:34:02.698182 19153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:02.699544 19132 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 22:34:02.699667 19132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-0/wal
uuid: "4750e9a921794581ae20ed0f1e6d7b4a"
format_stamp: "Formatted at 2026-08-08 22:34:02 on dist-test-slave-266d"
server_key: "8f4aa29cd9b03f0c8c26f3f4d4826658"
server_key_iv: "f02391f089d936fa398fa6b2e51bac47"
server_key_version: "kuduclusterkey@0"
I20260808 22:34:02.699800 19132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:34:02.713156 19132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:34:02.716269 19132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:34:02.716517 19132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:34:02.717262 19132 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:34:02.718303 19132 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:34:02.718407 19132 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:02.718529 19132 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:34:02.718612 19132 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:02.728871 19132 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.129:33795
I20260808 22:34:02.730010 19132 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
I20260808 22:34:02.738484 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 19132
I20260808 22:34:02.738608 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-0/wal/instance
I20260808 22:34:02.738940 32094 external_mini_cluster.cc:1428] Setting key a56088b6f39a1526a60cd9defea84c72
I20260808 22:34:02.746511 19312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.129:33795 every 8 connection(s)
WARNING: no policy specified for kudu/127.31.87.130@KRBTEST.COM; defaulting to no policy
Aug 08 22:34:02 dist-test-slave-266d krb5kdc[18310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228442, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.129@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.130@KRBTEST.COM" created.
I20260808 22:34:02.764824 19324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:02.747492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45947 (local address 127.31.87.129:41147)
0808 22:34:02.747991 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:02.748041 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 22:34:02.749006 (+ 965us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:34:02.749909 (+ 903us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:34:02.749919 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:34:02.750366 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:34:02.751013 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:02.751033 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:02.752231 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:02.752235 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:02.753520 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:02.753532 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:02.754372 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:02.754376 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:02.754588 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:02.755183 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:34:02.755205 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:34:02.759556 (+ 4351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:34:02.761789 (+ 2233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:02.761794 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:02.761809 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:02.762119 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:02.762428 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:02.762431 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:02.762433 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:02.762598 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:02.763077 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:34:02.763084 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:34:02.763363 (+ 279us) client_negotiation.cc:769] Sending connection context
0808 22:34:02.763546 (+ 183us) client_negotiation.cc:240] Negotiation successful
0808 22:34:02.763862 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":129,"threads_started":1}
I20260808 22:34:02.765630 19323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:02.747542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:41147 (local address 127.31.87.190:45947)
0808 22:34:02.747851 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:02.747861 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:02.749168 (+ 1307us) server_negotiation.cc:407] Connection header received
0808 22:34:02.749459 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:02.749463 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:02.749520 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:02.749604 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:34:02.751241 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:02.752067 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:02.753691 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:02.754065 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:02.754179 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:02.759756 (+ 5577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:34:02.759760 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:34:02.759763 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:34:02.759791 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:34:02.761640 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:02.762236 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:02.762239 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:02.762241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:02.762329 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:02.762706 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:02.762722 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:02.762724 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:02.762935 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:34:02.763022 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:02.765264 (+ 2242us) server_negotiation.cc:299] Negotiation successful
0808 22:34:02.765468 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":90,"threads_started":1}
I20260808 22:34:02.770467 19313 heartbeater.cc:347] Connected to a master server at 127.31.87.190:45947
I20260808 22:34:02.770709 19313 heartbeater.cc:464] Registering TS with master...
I20260808 22:34:02.771296 19313 heartbeater.cc:511] Master 127.31.87.190:45947 requested a full tablet report, sending...
I20260808 22:34:02.773020 19009 ts_manager.cc:194] Registered new tserver with Master: 4750e9a921794581ae20ed0f1e6d7b4a (127.31.87.129:33795)
I20260808 22:34:02.774569 19009 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.129@KRBTEST.COM'} at 127.31.87.129:41147
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab.
Entry for principal kudu/127.31.87.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab.
I20260808 22:34:02.780035 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.87.148:41391/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.130:0
--local_ip_for_outbound_sockets=127.31.87.130
--webserver_interface=127.31.87.130
--webserver_port=0
--tserver_master_addrs=127.31.87.190:45947
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab
--principal=kudu/127.31.87.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:38239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 22:34:02 dist-test-slave-266d krb5kdc[18310](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786228442, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 22:34:02.908414 19329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:34:02.908640 19329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:34:02.908696 19329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:34:02.912590 19329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:34:02.912640 19329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:34:02.912726 19329 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.130
I20260808 22:34:02.917768 19329 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:38239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.31.87.148:41391/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab
--principal=kudu/127.31.87.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.87.130
--webserver_port=0
--tserver_master_addrs=127.31.87.190:45947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19329
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.130
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:34:02.918677 19329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:34:02.919904 19329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:34:02.930007 19342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:34:02.930233 19339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:34:02.930033 19338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:34:02.930935 19329 server_base.cc:1034] running on GCE node
I20260808 22:34:02.931442 19329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:34:02.932029 19329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:34:02.933156 19329 hybrid_clock.cc:648] HybridClock initialized: now 1786228442933134 us; error 33 us; skew 500 ppm
Aug 08 22:34:02 dist-test-slave-266d krb5kdc[18310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228442, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:34:02.935973 19329 init.cc:377] Logged in from keytab as kudu/127.31.87.130@KRBTEST.COM (short username kudu)
I20260808 22:34:02.937063 19329 webserver.cc:460] Webserver started at http://127.31.87.130:35553/ using document root <none> and password file <none>
I20260808 22:34:02.937781 19329 fs_manager.cc:359] Metadata directory not provided
I20260808 22:34:02.937881 19329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:34:02.938199 19329 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 22:34:02 dist-test-slave-266d krb5kdc[18310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228442, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.130@KRBTEST.COM for HTTP/127.31.87.148@KRBTEST.COM
I20260808 22:34:02.962057 19329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-1/data/instance:
uuid: "f6108757b23a46dfb58cf5e7350f3e85"
format_stamp: "Formatted at 2026-08-08 22:34:02 on dist-test-slave-266d"
server_key: "be892f1bf60a4ea5f6b4db107c0d1685"
server_key_iv: "a83128201d4e596d781bb72196b0ede0"
server_key_version: "kuduclusterkey@0"
I20260808 22:34:02.962826 19329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-1/wal/instance:
uuid: "f6108757b23a46dfb58cf5e7350f3e85"
format_stamp: "Formatted at 2026-08-08 22:34:02 on dist-test-slave-266d"
server_key: "be892f1bf60a4ea5f6b4db107c0d1685"
server_key_iv: "a83128201d4e596d781bb72196b0ede0"
server_key_version: "kuduclusterkey@0"
I20260808 22:34:02.970546 19329 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260808 22:34:02.996227 19353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:03.006455 19329 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.002s sys 0.000s
I20260808 22:34:03.006711 19329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-1/wal
uuid: "f6108757b23a46dfb58cf5e7350f3e85"
format_stamp: "Formatted at 2026-08-08 22:34:02 on dist-test-slave-266d"
server_key: "be892f1bf60a4ea5f6b4db107c0d1685"
server_key_iv: "a83128201d4e596d781bb72196b0ede0"
server_key_version: "kuduclusterkey@0"
I20260808 22:34:03.006961 19329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:34:03.059134 19329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:34:03.066908 19329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:34:03.067201 19329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:34:03.067968 19329 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:34:03.069141 19329 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:34:03.069252 19329 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:03.069361 19329 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:34:03.069499 19329 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:03.080793 19329 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.130:41815
I20260808 22:34:03.082108 19329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
I20260808 22:34:03.086094 19515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.130:41815 every 8 connection(s)
I20260808 22:34:03.089951 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 19329
I20260808 22:34:03.090061 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-1/wal/instance
I20260808 22:34:03.090358 32094 external_mini_cluster.cc:1428] Setting key 94a30531dc20648fdc9ef13a56273caf
Aug 08 22:34:03 dist-test-slave-266d krb5kdc[18310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228442, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.130@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.87.131@KRBTEST.COM; defaulting to no policy
I20260808 22:34:03.110858 19323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:03.089340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:59751 (local address 127.31.87.190:45947)
0808 22:34:03.089471 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:03.089476 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:03.097621 (+ 8145us) server_negotiation.cc:407] Connection header received
0808 22:34:03.098788 (+ 1167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:03.098792 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:03.098848 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:03.098901 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:34:03.100523 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:03.101377 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:03.103109 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:03.103467 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:03.103549 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:03.106672 (+ 3123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:34:03.106676 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:34:03.106679 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:34:03.106707 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:34:03.108390 (+ 1683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:03.109022 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:03.109026 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:03.109028 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:03.109141 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:03.109546 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:03.109549 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:03.109551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:03.109782 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:34:03.109858 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:03.110533 (+ 675us) server_negotiation.cc:299] Negotiation successful
0808 22:34:03.110680 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.131@KRBTEST.COM" created.
I20260808 22:34:03.112422 19519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:03.089123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45947 (local address 127.31.87.130:59751)
0808 22:34:03.097423 (+ 8300us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:03.097485 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 22:34:03.098514 (+ 1029us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:34:03.099321 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:34:03.099331 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:34:03.099681 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:34:03.100348 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:03.100368 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:03.101536 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:03.101540 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:03.102964 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:03.102976 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:03.103773 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:03.103777 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:03.103974 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:03.104689 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:34:03.104711 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:34:03.106453 (+ 1742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:34:03.108545 (+ 2092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:03.108551 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:03.108578 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:03.108905 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:03.109257 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:03.109260 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:03.109262 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:03.109434 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:03.109945 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:34:03.109951 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:34:03.110252 (+ 301us) client_negotiation.cc:769] Sending connection context
0808 22:34:03.111035 (+ 783us) client_negotiation.cc:240] Negotiation successful
0808 22:34:03.111362 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8051,"thread_start_us":127,"threads_started":1}
I20260808 22:34:03.114916 19517 heartbeater.cc:347] Connected to a master server at 127.31.87.190:45947
I20260808 22:34:03.115178 19517 heartbeater.cc:464] Registering TS with master...
I20260808 22:34:03.115744 19517 heartbeater.cc:511] Master 127.31.87.190:45947 requested a full tablet report, sending...
I20260808 22:34:03.125860 19009 ts_manager.cc:194] Registered new tserver with Master: f6108757b23a46dfb58cf5e7350f3e85 (127.31.87.130:41815)
I20260808 22:34:03.126681 19009 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.130@KRBTEST.COM'} at 127.31.87.130:59751
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab.
Entry for principal kudu/127.31.87.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab.
I20260808 22:34:03.148327 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.87.148:41391/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.131:0
--local_ip_for_outbound_sockets=127.31.87.131
--webserver_interface=127.31.87.131
--webserver_port=0
--tserver_master_addrs=127.31.87.190:45947
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab
--principal=kudu/127.31.87.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:38239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:34:03.326190 19523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:34:03.326627 19523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:34:03.326742 19523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:34:03.333289 19523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:34:03.333427 19523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:34:03.333567 19523 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.131
Aug 08 22:34:03 dist-test-slave-266d krb5kdc[18310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228442, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for HTTP/127.31.87.148@KRBTEST.COM
I20260808 22:34:03.341194 19523 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:38239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.31.87.148:41391/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab
--principal=kudu/127.31.87.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.87.131
--webserver_port=0
--tserver_master_addrs=127.31.87.190:45947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19523
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.131
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:34:03.342980 19523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:34:03.344462 19523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:34:03.361917 19533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:34:03.361920 19531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:34:03.362677 19530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:34:03.365293 19523 server_base.cc:1034] running on GCE node
I20260808 22:34:03.365845 19523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:34:03.366638 19523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:34:03.373986 19523 hybrid_clock.cc:648] HybridClock initialized: now 1786228443373898 us; error 95 us; skew 500 ppm
Aug 08 22:34:03 dist-test-slave-266d krb5kdc[18310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228443, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:34:03.377617 19523 init.cc:377] Logged in from keytab as kudu/127.31.87.131@KRBTEST.COM (short username kudu)
I20260808 22:34:03.378728 19523 webserver.cc:460] Webserver started at http://127.31.87.131:38339/ using document root <none> and password file <none>
I20260808 22:34:03.379477 19523 fs_manager.cc:359] Metadata directory not provided
I20260808 22:34:03.379632 19523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:34:03.379909 19523 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 22:34:03 dist-test-slave-266d krb5kdc[18310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228443, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.131@KRBTEST.COM for HTTP/127.31.87.148@KRBTEST.COM
I20260808 22:34:03.398743 19523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-2/data/instance:
uuid: "b503ed7b0f3a4950b37c97e2b6f96b9e"
format_stamp: "Formatted at 2026-08-08 22:34:03 on dist-test-slave-266d"
server_key: "369f66862ba9b0427871c4f8170bb70b"
server_key_iv: "0377031a648c260247094527ccf67d51"
server_key_version: "kuduclusterkey@0"
I20260808 22:34:03.400728 19523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-2/wal/instance:
uuid: "b503ed7b0f3a4950b37c97e2b6f96b9e"
format_stamp: "Formatted at 2026-08-08 22:34:03 on dist-test-slave-266d"
server_key: "369f66862ba9b0427871c4f8170bb70b"
server_key_iv: "0377031a648c260247094527ccf67d51"
server_key_version: "kuduclusterkey@0"
I20260808 22:34:03.406428 19523 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260808 22:34:03.431573 19541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:03.433396 19523 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260808 22:34:03.433527 19523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-2/wal
uuid: "b503ed7b0f3a4950b37c97e2b6f96b9e"
format_stamp: "Formatted at 2026-08-08 22:34:03 on dist-test-slave-266d"
server_key: "369f66862ba9b0427871c4f8170bb70b"
server_key_iv: "0377031a648c260247094527ccf67d51"
server_key_version: "kuduclusterkey@0"
I20260808 22:34:03.433650 19523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:34:03.454098 19523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:34:03.457576 19523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:34:03.457816 19523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:34:03.458532 19523 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:34:03.459637 19523 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:34:03.459739 19523 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:03.459831 19523 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:34:03.459890 19523 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:03.473429 19523 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.131:43569
I20260808 22:34:03.474654 19523 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
I20260808 22:34:03.476372 19700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.131:43569 every 8 connection(s)
I20260808 22:34:03.484676 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 19523
I20260808 22:34:03.484807 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-2/wal/instance
I20260808 22:34:03.485150 32094 external_mini_cluster.cc:1428] Setting key 1cb54cac01839a68525beed23d219d21
Aug 08 22:34:03 dist-test-slave-266d krb5kdc[18310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228443, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.131@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:34:03.493835 19323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:03.478564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:47297 (local address 127.31.87.190:45947)
0808 22:34:03.478732 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:03.478749 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:03.480497 (+ 1748us) server_negotiation.cc:407] Connection header received
0808 22:34:03.481795 (+ 1298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:03.481799 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:03.481853 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:03.481904 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:34:03.483485 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:03.484356 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:03.486105 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:03.486462 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:03.486537 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:03.489699 (+ 3162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:34:03.489702 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:34:03.489705 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:34:03.489734 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:34:03.491445 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:03.492043 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:03.492047 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:03.492049 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:03.492136 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:03.492535 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:03.492538 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:03.492540 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:03.492752 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:34:03.492831 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:03.493519 (+ 688us) server_negotiation.cc:299] Negotiation successful
0808 22:34:03.493668 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 22:34:03.495922 19704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:03.479054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45947 (local address 127.31.87.131:47297)
0808 22:34:03.480336 (+ 1282us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:03.480391 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 22:34:03.481546 (+ 1155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:34:03.482311 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:34:03.482320 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:34:03.482671 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:34:03.483314 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:03.483332 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:03.484519 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:03.484522 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:03.485961 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:03.485973 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:03.486733 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:03.486736 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:03.486931 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:03.487714 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:34:03.487734 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:34:03.489509 (+ 1775us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:34:03.491584 (+ 2075us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:03.491589 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:03.491605 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:03.491927 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:03.492234 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:03.492248 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:03.492250 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:03.492430 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:03.492896 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:34:03.492903 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:34:03.493228 (+ 325us) client_negotiation.cc:769] Sending connection context
0808 22:34:03.494493 (+ 1265us) client_negotiation.cc:240] Negotiation successful
0808 22:34:03.494834 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1037,"thread_start_us":156,"threads_started":1}
I20260808 22:34:03.497462 19701 heartbeater.cc:347] Connected to a master server at 127.31.87.190:45947
I20260808 22:34:03.497737 19701 heartbeater.cc:464] Registering TS with master...
I20260808 22:34:03.498306 19701 heartbeater.cc:511] Master 127.31.87.190:45947 requested a full tablet report, sending...
I20260808 22:34:03.500888 19009 ts_manager.cc:194] Registered new tserver with Master: b503ed7b0f3a4950b37c97e2b6f96b9e (127.31.87.131:43569)
I20260808 22:34:03.501665 19009 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.131@KRBTEST.COM'} at 127.31.87.131:47297
I20260808 22:34:03.512181 32094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:34:03.513280 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 19132
I20260808 22:34:03.533056 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 19329
I20260808 22:34:03.555860 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 19523
I20260808 22:34:03.573601 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 18961
I20260808 22:34:03.586655 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:45947
--webserver_interface=127.31.87.190
--webserver_port=38249
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:38239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:45947
--ranger_config_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.87.148:41391/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.31.87.148:41391/kms with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:34:03.692955 19705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:34:03.693231 19705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:34:03.693295 19705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:34:03.697166 19705 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:34:03.697216 19705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:34:03.697248 19705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:34:03.697271 19705 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:34:03.697299 19705 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:34:03.703168 19705 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:38239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.31.87.148:41391/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:45947
--ranger_config_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:45947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.87.190
--webserver_port=38249
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19705
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:34:03.704066 19705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:34:03.704999 19705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:34:03.713037 19715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:34:03.713207 19718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:34:03.713116 19714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:34:03.713281 19705 server_base.cc:1034] running on GCE node
I20260808 22:34:03.713886 19705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:34:03.714881 19705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:34:03.716042 19705 hybrid_clock.cc:648] HybridClock initialized: now 1786228443716022 us; error 31 us; skew 500 ppm
Aug 08 22:34:03 dist-test-slave-266d krb5kdc[18310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228443, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:34:03.718813 19705 init.cc:377] Logged in from keytab as kudu/127.31.87.190@KRBTEST.COM (short username kudu)
I20260808 22:34:03.719678 19705 webserver.cc:460] Webserver started at http://127.31.87.190:38249/ using document root <none> and password file <none>
I20260808 22:34:03.720184 19705 fs_manager.cc:359] Metadata directory not provided
I20260808 22:34:03.720259 19705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 22:34:03 dist-test-slave-266d krb5kdc[18310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228443, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for HTTP/127.31.87.148@KRBTEST.COM
I20260808 22:34:03.758257 19705 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.002s
I20260808 22:34:03.760545 19730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:03.761883 19705 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.002s
I20260808 22:34:03.762003 19705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "ddb477d3689f4bbb812a0b93b43c1673"
format_stamp: "Formatted at 2026-08-08 22:34:01 on dist-test-slave-266d"
server_key: "dc3c88f7156b2423d3f45a9212bc30ad"
server_key_iv: "aabd6a2104c0c08ee7e0f42b11a4cb7a"
server_key_version: "kuduclusterkey@0"
I20260808 22:34:03.762531 19705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:34:03.776265 19705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:34:03.779080 19705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:34:03.779259 19705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:34:03.786001 19705 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.190:45947
I20260808 22:34:03.786013 19802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.190:45947 every 8 connection(s)
I20260808 22:34:03.786985 19705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:34:03.791152 19804 sys_catalog.cc:263] Verifying existing consensus state
I20260808 22:34:03.792723 19804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddb477d3689f4bbb812a0b93b43c1673: Bootstrap starting.
I20260808 22:34:03.793273 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 19705
I20260808 22:34:03.793702 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.87.148:41391/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.129:33795
--local_ip_for_outbound_sockets=127.31.87.129
--tserver_master_addrs=127.31.87.190:45947
--webserver_port=46065
--webserver_interface=127.31.87.129
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab
--principal=kudu/127.31.87.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:38239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.31.87.148:41391/kms with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 22:34:03.808554 19804 log.cc:824] T 00000000000000000000000000000000 P ddb477d3689f4bbb812a0b93b43c1673: Log is configured to *not* fsync() on all Append() calls
I20260808 22:34:03.815021 19804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddb477d3689f4bbb812a0b93b43c1673: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 22:34:03.815364 19804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddb477d3689f4bbb812a0b93b43c1673: Bootstrap complete.
I20260808 22:34:03.819198 19804 raft_consensus.cc:348] T 00000000000000000000000000000000 P ddb477d3689f4bbb812a0b93b43c1673 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddb477d3689f4bbb812a0b93b43c1673" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45947 } }
I20260808 22:34:03.819573 19804 raft_consensus.cc:729] T 00000000000000000000000000000000 P ddb477d3689f4bbb812a0b93b43c1673 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddb477d3689f4bbb812a0b93b43c1673, State: Initialized, Role: FOLLOWER
I20260808 22:34:03.820014 19804 consensus_queue.cc:260] T 00000000000000000000000000000000 P ddb477d3689f4bbb812a0b93b43c1673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddb477d3689f4bbb812a0b93b43c1673" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45947 } }
I20260808 22:34:03.820132 19804 raft_consensus.cc:388] T 00000000000000000000000000000000 P ddb477d3689f4bbb812a0b93b43c1673 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:34:03.820195 19804 raft_consensus.cc:482] T 00000000000000000000000000000000 P ddb477d3689f4bbb812a0b93b43c1673 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:34:03.820300 19804 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ddb477d3689f4bbb812a0b93b43c1673 [term 1 FOLLOWER]: Advancing to term 2
I20260808 22:34:03.822198 19804 raft_consensus.cc:504] T 00000000000000000000000000000000 P ddb477d3689f4bbb812a0b93b43c1673 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddb477d3689f4bbb812a0b93b43c1673" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45947 } }
I20260808 22:34:03.822520 19804 leader_election.cc:302] T 00000000000000000000000000000000 P ddb477d3689f4bbb812a0b93b43c1673 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ddb477d3689f4bbb812a0b93b43c1673; no voters:
I20260808 22:34:03.822798 19804 leader_election.cc:288] T 00000000000000000000000000000000 P ddb477d3689f4bbb812a0b93b43c1673 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 22:34:03.822889 19811 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ddb477d3689f4bbb812a0b93b43c1673 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 22:34:03.823138 19811 raft_consensus.cc:686] T 00000000000000000000000000000000 P ddb477d3689f4bbb812a0b93b43c1673 [term 2 LEADER]: Becoming Leader. State: Replica: ddb477d3689f4bbb812a0b93b43c1673, State: Running, Role: LEADER
I20260808 22:34:03.823475 19811 consensus_queue.cc:237] T 00000000000000000000000000000000 P ddb477d3689f4bbb812a0b93b43c1673 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddb477d3689f4bbb812a0b93b43c1673" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45947 } }
I20260808 22:34:03.824002 19804 sys_catalog.cc:564] T 00000000000000000000000000000000 P ddb477d3689f4bbb812a0b93b43c1673 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:34:03.824786 19812 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddb477d3689f4bbb812a0b93b43c1673 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ddb477d3689f4bbb812a0b93b43c1673" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddb477d3689f4bbb812a0b93b43c1673" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45947 } } }
I20260808 22:34:03.824913 19812 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddb477d3689f4bbb812a0b93b43c1673 [sys.catalog]: This master's current role is: LEADER
I20260808 22:34:03.825240 19813 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddb477d3689f4bbb812a0b93b43c1673 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ddb477d3689f4bbb812a0b93b43c1673. Latest consensus state: current_term: 2 leader_uuid: "ddb477d3689f4bbb812a0b93b43c1673" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddb477d3689f4bbb812a0b93b43c1673" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45947 } } }
I20260808 22:34:03.825383 19813 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddb477d3689f4bbb812a0b93b43c1673 [sys.catalog]: This master's current role is: LEADER
I20260808 22:34:03.825736 19823 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:34:03.826180 19804 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 22:34:03.832046 19823 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:34:03.832894 19823 catalog_manager.cc:1259] Loaded cluster ID: 2116a155a15b44bcacf0a9a9f47fff39
I20260808 22:34:03.832990 19823 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:34:03.835526 19823 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:34:03.836258 19823 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ddb477d3689f4bbb812a0b93b43c1673: Loaded TSK: 0
I20260808 22:34:03.837565 19823 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 22:34:03.935786 19808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:34:03.936091 19808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:34:03.936152 19808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:34:03.942090 19808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:34:03.942162 19808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:34:03.942242 19808 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.129
I20260808 22:34:03.949483 19808 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:38239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.31.87.148:41391/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab
--principal=kudu/127.31.87.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.129:33795
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.87.129
--webserver_port=46065
--tserver_master_addrs=127.31.87.190:45947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19808
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.129
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:34:03.950790 19808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:34:03.952071 19808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:34:03.964599 19859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:34:03.964818 19808 server_base.cc:1034] running on GCE node
W20260808 22:34:03.964641 19864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:34:03.969259 19860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:34:03.969789 19808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:34:03.970656 19808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:34:03.971944 19808 hybrid_clock.cc:648] HybridClock initialized: now 1786228443971867 us; error 92 us; skew 500 ppm
Aug 08 22:34:03 dist-test-slave-266d krb5kdc[18310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228443, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:34:03.978417 19808 init.cc:377] Logged in from keytab as kudu/127.31.87.129@KRBTEST.COM (short username kudu)
I20260808 22:34:03.979615 19808 webserver.cc:460] Webserver started at http://127.31.87.129:46065/ using document root <none> and password file <none>
I20260808 22:34:03.980494 19808 fs_manager.cc:359] Metadata directory not provided
I20260808 22:34:03.980662 19808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 22:34:03 dist-test-slave-266d krb5kdc[18310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228443, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.129@KRBTEST.COM for HTTP/127.31.87.148@KRBTEST.COM
I20260808 22:34:04.007045 19808 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 22:34:04.010507 19875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:04.012143 19808 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260808 22:34:04.012260 19808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-0/wal
uuid: "4750e9a921794581ae20ed0f1e6d7b4a"
format_stamp: "Formatted at 2026-08-08 22:34:02 on dist-test-slave-266d"
server_key: "8f4aa29cd9b03f0c8c26f3f4d4826658"
server_key_iv: "f02391f089d936fa398fa6b2e51bac47"
server_key_version: "kuduclusterkey@0"
I20260808 22:34:04.012928 19808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:34:04.035943 19808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:34:04.039397 19808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:34:04.039654 19808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:34:04.040421 19808 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:34:04.041468 19808 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:34:04.041574 19808 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:04.041707 19808 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:34:04.041813 19808 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:04.052488 19808 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.129:33795
I20260808 22:34:04.052531 20038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.129:33795 every 8 connection(s)
I20260808 22:34:04.053850 19808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
I20260808 22:34:04.059571 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 19808
I20260808 22:34:04.059993 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.87.148:41391/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.130:41815
--local_ip_for_outbound_sockets=127.31.87.130
--tserver_master_addrs=127.31.87.190:45947
--webserver_port=35553
--webserver_interface=127.31.87.130
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab
--principal=kudu/127.31.87.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:38239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.31.87.148:41391/kms with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 22:34:04 dist-test-slave-266d krb5kdc[18310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228443, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.129@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:34:04.087366 20044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:04.057123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45947 (local address 127.31.87.129:42427)
0808 22:34:04.061422 (+ 4299us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:04.061485 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 22:34:04.062592 (+ 1107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:34:04.066193 (+ 3601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:34:04.066202 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:34:04.066632 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:34:04.067325 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:04.067349 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:04.069316 (+ 1967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:04.069320 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:04.070711 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:04.070724 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:04.072059 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:04.072064 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:04.072262 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:04.072997 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:34:04.073020 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:34:04.074644 (+ 1624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:34:04.080196 (+ 5552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:04.080203 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:04.080219 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:04.080596 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:04.083994 (+ 3398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:04.083998 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:04.084001 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:04.084303 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:04.085476 (+ 1173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:34:04.085483 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:34:04.085805 (+ 322us) client_negotiation.cc:769] Sending connection context
0808 22:34:04.086010 (+ 205us) client_negotiation.cc:240] Negotiation successful
0808 22:34:04.086343 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4057,"thread_start_us":164,"threads_started":1}
I20260808 22:34:04.089871 20045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:04.058553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:42427 (local address 127.31.87.190:45947)
0808 22:34:04.058980 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:04.059001 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:04.064781 (+ 5780us) server_negotiation.cc:407] Connection header received
0808 22:34:04.065167 (+ 386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:04.065185 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:04.065667 (+ 482us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:04.066122 (+ 455us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:34:04.067553 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:04.069131 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:04.070865 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:04.071638 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:04.071817 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:04.074878 (+ 3061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:34:04.074886 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:34:04.074904 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:34:04.074941 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:34:04.080020 (+ 5079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:04.083650 (+ 3630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:04.083666 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:04.083672 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:04.083847 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:04.084496 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:04.084500 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:04.084502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:04.085329 (+ 827us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:34:04.088429 (+ 3100us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:04.088602 (+ 173us) server_negotiation.cc:299] Negotiation successful
0808 22:34:04.088922 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":99,"threads_started":1}
I20260808 22:34:04.092939 20039 heartbeater.cc:347] Connected to a master server at 127.31.87.190:45947
I20260808 22:34:04.093216 20039 heartbeater.cc:464] Registering TS with master...
I20260808 22:34:04.093822 20039 heartbeater.cc:511] Master 127.31.87.190:45947 requested a full tablet report, sending...
W20260808 22:34:04.096176 20039 heartbeater.cc:644] Failed to heartbeat to 127.31.87.190:45947 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 22:34:04.096303 20039 heartbeater.cc:464] Registering TS with master...
I20260808 22:34:04.096499 20039 heartbeater.cc:511] Master 127.31.87.190:45947 requested a full tablet report, sending...
I20260808 22:34:04.097584 20039 heartbeater.cc:464] Registering TS with master...
I20260808 22:34:04.097774 20039 heartbeater.cc:511] Master 127.31.87.190:45947 requested a full tablet report, sending...
W20260808 22:34:04.098685 20039 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 22:34:04.192451 20046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:34:04.192715 20046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:34:04.192782 20046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:34:04.196966 20046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:34:04.197026 20046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:34:04.197116 20046 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.130
I20260808 22:34:04.203759 20046 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:38239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.31.87.148:41391/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab
--principal=kudu/127.31.87.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.130:41815
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.87.130
--webserver_port=35553
--tserver_master_addrs=127.31.87.190:45947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20046
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.130
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:34:04.205128 20046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:34:04.206442 20046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:34:04.218971 20053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:34:04.219607 20057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:34:04.220569 20054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:34:04.221873 20046 server_base.cc:1034] running on GCE node
I20260808 22:34:04.222473 20046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:34:04.223254 20046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:34:04.224452 20046 hybrid_clock.cc:648] HybridClock initialized: now 1786228444224416 us; error 49 us; skew 500 ppm
Aug 08 22:34:04 dist-test-slave-266d krb5kdc[18310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228444, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:34:04.228012 20046 init.cc:377] Logged in from keytab as kudu/127.31.87.130@KRBTEST.COM (short username kudu)
I20260808 22:34:04.229132 20046 webserver.cc:460] Webserver started at http://127.31.87.130:35553/ using document root <none> and password file <none>
I20260808 22:34:04.229885 20046 fs_manager.cc:359] Metadata directory not provided
I20260808 22:34:04.229998 20046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 22:34:04 dist-test-slave-266d krb5kdc[18310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228444, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.130@KRBTEST.COM for HTTP/127.31.87.148@KRBTEST.COM
I20260808 22:34:04.260524 20046 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.003s
I20260808 22:34:04.263828 20066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:04.265503 20046 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260808 22:34:04.265650 20046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-1/wal
uuid: "f6108757b23a46dfb58cf5e7350f3e85"
format_stamp: "Formatted at 2026-08-08 22:34:02 on dist-test-slave-266d"
server_key: "be892f1bf60a4ea5f6b4db107c0d1685"
server_key_iv: "a83128201d4e596d781bb72196b0ede0"
server_key_version: "kuduclusterkey@0"
I20260808 22:34:04.266362 20046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:34:04.295830 20046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:34:04.299597 20046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:34:04.299863 20046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:34:04.300590 20046 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:34:04.301762 20046 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:34:04.301875 20046 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:04.301985 20046 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:34:04.302067 20046 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:04.312701 20046 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.130:41815
I20260808 22:34:04.312819 20225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.130:41815 every 8 connection(s)
I20260808 22:34:04.314064 20046 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
I20260808 22:34:04.320678 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 20046
I20260808 22:34:04.321197 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.87.148:41391/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.131:43569
--local_ip_for_outbound_sockets=127.31.87.131
--tserver_master_addrs=127.31.87.190:45947
--webserver_port=38339
--webserver_interface=127.31.87.131
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab
--principal=kudu/127.31.87.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:38239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.31.87.148:41391/kms with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 22:34:04 dist-test-slave-266d krb5kdc[18310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228444, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.130@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:34:04.332690 20045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:04.317754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:33195 (local address 127.31.87.190:45947)
0808 22:34:04.317934 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:04.317939 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:04.318402 (+ 463us) server_negotiation.cc:407] Connection header received
0808 22:34:04.319467 (+ 1065us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:04.319471 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:04.319534 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:04.319634 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:34:04.321058 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:04.321917 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:04.323051 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:04.323294 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:04.323387 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:04.328921 (+ 5534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:34:04.328925 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:34:04.328927 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:34:04.328954 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:34:04.330487 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:04.331107 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:04.331111 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:04.331113 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:04.331210 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:04.331602 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:04.331604 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:04.331605 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:04.331784 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:34:04.331872 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:04.332383 (+ 511us) server_negotiation.cc:299] Negotiation successful
0808 22:34:04.332512 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260808 22:34:04.333560 20230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:04.317674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45947 (local address 127.31.87.130:33195)
0808 22:34:04.318215 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:04.318288 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0808 22:34:04.319234 (+ 946us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:34:04.319835 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:34:04.319844 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:34:04.320206 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:34:04.320872 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:04.320894 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:04.322076 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:04.322078 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:04.322929 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:04.322938 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:04.323542 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:04.323544 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:04.323701 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:04.324301 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:34:04.324318 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:34:04.328701 (+ 4383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:34:04.330601 (+ 1900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:04.330607 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:04.330623 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:04.330966 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:04.331334 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:04.331336 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:04.331338 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:04.331498 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:04.331889 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:34:04.331896 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:34:04.332153 (+ 257us) client_negotiation.cc:769] Sending connection context
0808 22:34:04.332325 (+ 172us) client_negotiation.cc:240] Negotiation successful
0808 22:34:04.332638 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":141,"threads_started":1}
I20260808 22:34:04.334790 20226 heartbeater.cc:347] Connected to a master server at 127.31.87.190:45947
I20260808 22:34:04.335026 20226 heartbeater.cc:464] Registering TS with master...
I20260808 22:34:04.335501 20226 heartbeater.cc:511] Master 127.31.87.190:45947 requested a full tablet report, sending...
W20260808 22:34:04.336994 20226 heartbeater.cc:644] Failed to heartbeat to 127.31.87.190:45947 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 22:34:04.337146 20226 heartbeater.cc:464] Registering TS with master...
I20260808 22:34:04.337311 20226 heartbeater.cc:511] Master 127.31.87.190:45947 requested a full tablet report, sending...
I20260808 22:34:04.338122 20226 heartbeater.cc:464] Registering TS with master...
I20260808 22:34:04.338282 20226 heartbeater.cc:511] Master 127.31.87.190:45947 requested a full tablet report, sending...
W20260808 22:34:04.339027 20226 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 22:34:04.490413 20231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:34:04.490772 20231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:34:04.490844 20231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:34:04.496682 20231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:34:04.496767 20231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:34:04.496855 20231 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.131
I20260808 22:34:04.504598 20231 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:38239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.31.87.148:41391/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab
--principal=kudu/127.31.87.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.131:43569
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.87.131
--webserver_port=38339
--tserver_master_addrs=127.31.87.190:45947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20231
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.131
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:34:04.506006 20231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:34:04.507373 20231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:34:04.520025 20238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:34:04.520208 20231 server_base.cc:1034] running on GCE node
W20260808 22:34:04.520044 20242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:34:04.520048 20240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:34:04.520890 20231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:34:04.521708 20231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:34:04.522895 20231 hybrid_clock.cc:648] HybridClock initialized: now 1786228444522866 us; error 43 us; skew 500 ppm
Aug 08 22:34:04 dist-test-slave-266d krb5kdc[18310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228444, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:34:04.526661 20231 init.cc:377] Logged in from keytab as kudu/127.31.87.131@KRBTEST.COM (short username kudu)
I20260808 22:34:04.527776 20231 webserver.cc:460] Webserver started at http://127.31.87.131:38339/ using document root <none> and password file <none>
I20260808 22:34:04.528503 20231 fs_manager.cc:359] Metadata directory not provided
I20260808 22:34:04.528667 20231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 22:34:04 dist-test-slave-266d krb5kdc[18310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228444, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.131@KRBTEST.COM for HTTP/127.31.87.148@KRBTEST.COM
I20260808 22:34:04.548009 20231 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 22:34:04.554327 20254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:04.555648 20231 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.000s
I20260808 22:34:04.555760 20231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-2/wal
uuid: "b503ed7b0f3a4950b37c97e2b6f96b9e"
format_stamp: "Formatted at 2026-08-08 22:34:03 on dist-test-slave-266d"
server_key: "369f66862ba9b0427871c4f8170bb70b"
server_key_iv: "0377031a648c260247094527ccf67d51"
server_key_version: "kuduclusterkey@0"
I20260808 22:34:04.556286 20231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:34:04.574743 20231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:34:04.578819 20231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:34:04.579001 20231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:34:04.579641 20231 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:34:04.580812 20231 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:34:04.580873 20231 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:04.580958 20231 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:34:04.580992 20231 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:04.591327 20231 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.131:43569
I20260808 22:34:04.591390 20418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.131:43569 every 8 connection(s)
I20260808 22:34:04.592406 20231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
Aug 08 22:34:04 dist-test-slave-266d krb5kdc[18310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228444, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.131@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:34:04.602001 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 20231
I20260808 22:34:04.606323 20045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:04.595698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:47471 (local address 127.31.87.190:45947)
0808 22:34:04.595818 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:04.595822 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:04.595839 (+ 17us) server_negotiation.cc:407] Connection header received
0808 22:34:04.596467 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:04.596471 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:04.596529 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:04.596611 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:34:04.597705 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:04.598223 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:04.599329 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:04.599552 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:04.599652 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:04.602225 (+ 2573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:34:04.602230 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:34:04.602232 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:34:04.602261 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:34:04.604058 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:04.604552 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:04.604556 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:04.604558 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:04.604659 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:04.605045 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:04.605049 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:04.605051 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:04.605300 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:34:04.605403 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:04.606075 (+ 672us) server_negotiation.cc:299] Negotiation successful
0808 22:34:04.606203 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 22:34:04.607429 20421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:04.594699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45947 (local address 127.31.87.131:47471)
0808 22:34:04.595171 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:04.595224 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 22:34:04.596239 (+ 1015us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:34:04.596791 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:34:04.596798 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:34:04.597054 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:34:04.597557 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:04.597571 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:04.598344 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:04.598347 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:04.599208 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:04.599217 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:04.599687 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:04.599692 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:04.599851 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:04.600449 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:34:04.600466 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:34:04.602042 (+ 1576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:34:04.604184 (+ 2142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:04.604191 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:04.604206 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:04.604448 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:04.604771 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:04.604774 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:04.604776 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:04.604948 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:04.605464 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:34:04.605471 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:34:04.605783 (+ 312us) client_negotiation.cc:769] Sending connection context
0808 22:34:04.606009 (+ 226us) client_negotiation.cc:240] Negotiation successful
0808 22:34:04.606342 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":108,"threads_started":1}
I20260808 22:34:04.608927 20419 heartbeater.cc:347] Connected to a master server at 127.31.87.190:45947
I20260808 22:34:04.609194 20419 heartbeater.cc:464] Registering TS with master...
I20260808 22:34:04.609747 20419 heartbeater.cc:511] Master 127.31.87.190:45947 requested a full tablet report, sending...
I20260808 22:34:04.611258 19756 ts_manager.cc:194] Registered new tserver with Master: b503ed7b0f3a4950b37c97e2b6f96b9e (127.31.87.131:43569)
I20260808 22:34:04.612823 19756 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.131@KRBTEST.COM'} at 127.31.87.131:47471
I20260808 22:34:04.615419 20422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:04.602691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44950 (local address 127.31.87.190:45947)
0808 22:34:04.602959 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:04.602967 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:04.603179 (+ 212us) server_negotiation.cc:407] Connection header received
0808 22:34:04.603328 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:04.603331 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:04.603383 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:04.603470 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:34:04.604558 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:04.605268 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:04.606515 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:04.606853 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:04.606945 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:04.608405 (+ 1460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:34:04.608409 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:34:04.608412 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:34:04.608436 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:34:04.610087 (+ 1651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:04.613578 (+ 3491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:04.613582 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:04.613583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:04.613679 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:04.614011 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:04.614014 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:04.614016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:04.614241 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:34:04.614423 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:04.615120 (+ 697us) server_negotiation.cc:299] Negotiation successful
0808 22:34:04.615248 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":105,"threads_started":1}
Aug 08 22:34:04 dist-test-slave-266d krb5kdc[18310](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786228444, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:34:05.098908 20039 heartbeater.cc:464] Registering TS with master...
I20260808 22:34:05.099241 20039 heartbeater.cc:511] Master 127.31.87.190:45947 requested a full tablet report, sending...
I20260808 22:34:05.100008 19756 ts_manager.cc:194] Registered new tserver with Master: 4750e9a921794581ae20ed0f1e6d7b4a (127.31.87.129:33795)
I20260808 22:34:05.100648 19756 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.129@KRBTEST.COM'} at 127.31.87.129:42427
Aug 08 22:34:05 dist-test-slave-266d krb5kdc[18310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228444, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for HTTP/127.31.87.148@KRBTEST.COM
I20260808 22:34:05.339246 20226 heartbeater.cc:464] Registering TS with master...
I20260808 22:34:05.339524 20226 heartbeater.cc:511] Master 127.31.87.190:45947 requested a full tablet report, sending...
I20260808 22:34:05.340242 19756 ts_manager.cc:194] Registered new tserver with Master: f6108757b23a46dfb58cf5e7350f3e85 (127.31.87.130:41815)
I20260808 22:34:05.341010 19756 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.130@KRBTEST.COM'} at 127.31.87.130:33195
I20260808 22:34:05.347527 32094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:34:05.360106 20438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:05.349455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44956 (local address 127.31.87.190:45947)
0808 22:34:05.349937 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:05.349943 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:05.350184 (+ 241us) server_negotiation.cc:407] Connection header received
0808 22:34:05.350381 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:05.350384 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:05.350438 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:05.350503 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:34:05.351551 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:05.352403 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:05.353707 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:05.354081 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:05.354163 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:05.356051 (+ 1888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:34:05.356056 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:34:05.356058 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:34:05.356088 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:34:05.357948 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:05.358509 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:05.358513 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:05.358515 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:05.358626 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:05.358975 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:05.358979 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:05.358981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:05.359215 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:34:05.359334 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:05.359760 (+ 426us) server_negotiation.cc:299] Negotiation successful
0808 22:34:05.359900 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":69,"threads_started":1}
I20260808 22:34:05.365228 19756 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44956:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:34:05.370306 19756 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:34:05.398571 20444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:05.385936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:33795 (local address 127.0.0.1:52764)
0808 22:34:05.391856 (+ 5920us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:05.391910 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 22:34:05.392063 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:34:05.393558 (+ 1495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:34:05.393564 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:34:05.393589 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:34:05.393730 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:05.393737 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:05.395249 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:05.395253 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:05.397229 (+ 1976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:05.397239 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:05.398035 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:05.398038 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:05.398153 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:05.398193 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 22:34:05.398289 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 22:34:05.398410 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5737,"thread_start_us":85,"threads_started":1}
I20260808 22:34:05.399055 20446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:05.387049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52764 (local address 127.31.87.129:33795)
0808 22:34:05.391580 (+ 4531us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:05.391588 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:05.392035 (+ 447us) server_negotiation.cc:407] Connection header received
0808 22:34:05.392245 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:05.392251 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:05.392473 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:05.392628 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:34:05.393869 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:05.395114 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:05.397365 (+ 2251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:05.397910 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:05.398009 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:05.398162 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:05.398807 (+ 645us) server_negotiation.cc:299] Negotiation successful
0808 22:34:05.398913 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4404,"thread_start_us":128,"threads_started":1}
I20260808 22:34:05.402598 19940 tablet_service.cc:1467] Processing CreateTablet for tablet f4de241291da499bac2d097e93326414 (DEFAULT_TABLE table=test-table [id=08d93ced49944b67a14638225f91cbe7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:34:05.403797 19940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4de241291da499bac2d097e93326414. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:34:05.411128 20444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:05.388359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.130:41815 (local address 127.0.0.1:56870)
0808 22:34:05.399473 (+ 11114us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:05.399485 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:34:05.399551 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:34:05.402635 (+ 3084us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:34:05.402638 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:34:05.402656 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:34:05.402725 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:05.402731 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:05.404328 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:05.404332 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:05.405373 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:05.405384 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:05.410740 (+ 5356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:05.410744 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:05.410830 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:05.410855 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 22:34:05.410926 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 22:34:05.410990 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11058,"thread_start_us":99,"threads_started":1}
I20260808 22:34:05.412983 20450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:05.388937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40268 (local address 127.31.87.131:43569)
0808 22:34:05.389290 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:05.389298 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:05.403021 (+ 13723us) server_negotiation.cc:407] Connection header received
0808 22:34:05.405659 (+ 2638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:05.405665 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:05.405853 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:05.405965 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:34:05.406364 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:05.408169 (+ 1805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:05.411398 (+ 3229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:05.412187 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:05.412269 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:05.412392 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:05.412740 (+ 348us) server_negotiation.cc:299] Negotiation successful
0808 22:34:05.412840 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":135,"threads_started":1}
I20260808 22:34:05.413365 20447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:05.388716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.131:43569 (local address 127.0.0.1:40268)
0808 22:34:05.402923 (+ 14207us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:05.402934 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:34:05.405508 (+ 2574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:34:05.406149 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:34:05.406152 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:34:05.406169 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:34:05.406246 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:05.406251 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:05.408320 (+ 2069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:05.408323 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:05.409406 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:05.409417 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:05.412490 (+ 3073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:05.412494 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:05.412589 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:05.412608 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 22:34:05.413162 (+ 554us) client_negotiation.cc:240] Negotiation successful
0808 22:34:05.413240 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14150,"thread_start_us":84,"threads_started":1}
I20260808 22:34:05.414578 20448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:05.388516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56870 (local address 127.31.87.130:41815)
0808 22:34:05.402040 (+ 13524us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:05.402047 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:05.402071 (+ 24us) server_negotiation.cc:407] Connection header received
0808 22:34:05.402153 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:05.402159 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:05.402353 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:05.402493 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:34:05.403114 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:05.404199 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:05.409623 (+ 5424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:05.410420 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:05.410525 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:05.410671 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:05.414341 (+ 3670us) server_negotiation.cc:299] Negotiation successful
0808 22:34:05.414441 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13416,"thread_start_us":108,"threads_started":1}
I20260808 22:34:05.417083 20105 tablet_service.cc:1467] Processing CreateTablet for tablet f4de241291da499bac2d097e93326414 (DEFAULT_TABLE table=test-table [id=08d93ced49944b67a14638225f91cbe7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:34:05.416965 20331 tablet_service.cc:1467] Processing CreateTablet for tablet f4de241291da499bac2d097e93326414 (DEFAULT_TABLE table=test-table [id=08d93ced49944b67a14638225f91cbe7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:34:05.418090 20105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4de241291da499bac2d097e93326414. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:34:05.418090 20331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4de241291da499bac2d097e93326414. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:34:05.421429 20451 tablet_bootstrap.cc:492] T f4de241291da499bac2d097e93326414 P 4750e9a921794581ae20ed0f1e6d7b4a: Bootstrap starting.
I20260808 22:34:05.423820 20451 tablet_bootstrap.cc:654] T f4de241291da499bac2d097e93326414 P 4750e9a921794581ae20ed0f1e6d7b4a: Neither blocks nor log segments found. Creating new log.
I20260808 22:34:05.426849 20451 log.cc:824] T f4de241291da499bac2d097e93326414 P 4750e9a921794581ae20ed0f1e6d7b4a: Log is configured to *not* fsync() on all Append() calls
I20260808 22:34:05.430326 20452 tablet_bootstrap.cc:492] T f4de241291da499bac2d097e93326414 P f6108757b23a46dfb58cf5e7350f3e85: Bootstrap starting.
I20260808 22:34:05.432763 20452 tablet_bootstrap.cc:654] T f4de241291da499bac2d097e93326414 P f6108757b23a46dfb58cf5e7350f3e85: Neither blocks nor log segments found. Creating new log.
I20260808 22:34:05.433621 20452 log.cc:824] T f4de241291da499bac2d097e93326414 P f6108757b23a46dfb58cf5e7350f3e85: Log is configured to *not* fsync() on all Append() calls
I20260808 22:34:05.434274 20453 tablet_bootstrap.cc:492] T f4de241291da499bac2d097e93326414 P b503ed7b0f3a4950b37c97e2b6f96b9e: Bootstrap starting.
I20260808 22:34:05.436322 20453 tablet_bootstrap.cc:654] T f4de241291da499bac2d097e93326414 P b503ed7b0f3a4950b37c97e2b6f96b9e: Neither blocks nor log segments found. Creating new log.
I20260808 22:34:05.437165 20453 log.cc:824] T f4de241291da499bac2d097e93326414 P b503ed7b0f3a4950b37c97e2b6f96b9e: Log is configured to *not* fsync() on all Append() calls
I20260808 22:34:05.438578 20451 tablet_bootstrap.cc:492] T f4de241291da499bac2d097e93326414 P 4750e9a921794581ae20ed0f1e6d7b4a: No bootstrap required, opened a new log
I20260808 22:34:05.438778 20451 ts_tablet_manager.cc:1397] T f4de241291da499bac2d097e93326414 P 4750e9a921794581ae20ed0f1e6d7b4a: Time spent bootstrapping tablet: real 0.018s user 0.006s sys 0.000s
I20260808 22:34:05.442658 20451 raft_consensus.cc:348] T f4de241291da499bac2d097e93326414 P 4750e9a921794581ae20ed0f1e6d7b4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4750e9a921794581ae20ed0f1e6d7b4a" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 33795 } } peers { permanent_uuid: "b503ed7b0f3a4950b37c97e2b6f96b9e" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 43569 } } peers { permanent_uuid: "f6108757b23a46dfb58cf5e7350f3e85" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41815 } }
I20260808 22:34:05.442903 20451 raft_consensus.cc:374] T f4de241291da499bac2d097e93326414 P 4750e9a921794581ae20ed0f1e6d7b4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:34:05.442942 20451 raft_consensus.cc:729] T f4de241291da499bac2d097e93326414 P 4750e9a921794581ae20ed0f1e6d7b4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4750e9a921794581ae20ed0f1e6d7b4a, State: Initialized, Role: FOLLOWER
I20260808 22:34:05.443537 20451 consensus_queue.cc:260] T f4de241291da499bac2d097e93326414 P 4750e9a921794581ae20ed0f1e6d7b4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4750e9a921794581ae20ed0f1e6d7b4a" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 33795 } } peers { permanent_uuid: "b503ed7b0f3a4950b37c97e2b6f96b9e" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 43569 } } peers { permanent_uuid: "f6108757b23a46dfb58cf5e7350f3e85" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41815 } }
I20260808 22:34:05.444661 20451 ts_tablet_manager.cc:1428] T f4de241291da499bac2d097e93326414 P 4750e9a921794581ae20ed0f1e6d7b4a: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 22:34:05.445041 20039 heartbeater.cc:503] Master 127.31.87.190:45947 was elected leader, sending a full tablet report...
I20260808 22:34:05.447052 20452 tablet_bootstrap.cc:492] T f4de241291da499bac2d097e93326414 P f6108757b23a46dfb58cf5e7350f3e85: No bootstrap required, opened a new log
I20260808 22:34:05.447242 20452 ts_tablet_manager.cc:1397] T f4de241291da499bac2d097e93326414 P f6108757b23a46dfb58cf5e7350f3e85: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.000s
I20260808 22:34:05.450984 20452 raft_consensus.cc:348] T f4de241291da499bac2d097e93326414 P f6108757b23a46dfb58cf5e7350f3e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4750e9a921794581ae20ed0f1e6d7b4a" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 33795 } } peers { permanent_uuid: "b503ed7b0f3a4950b37c97e2b6f96b9e" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 43569 } } peers { permanent_uuid: "f6108757b23a46dfb58cf5e7350f3e85" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41815 } }
I20260808 22:34:05.451305 20452 raft_consensus.cc:374] T f4de241291da499bac2d097e93326414 P f6108757b23a46dfb58cf5e7350f3e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:34:05.451395 20452 raft_consensus.cc:729] T f4de241291da499bac2d097e93326414 P f6108757b23a46dfb58cf5e7350f3e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6108757b23a46dfb58cf5e7350f3e85, State: Initialized, Role: FOLLOWER
I20260808 22:34:05.451975 20452 consensus_queue.cc:260] T f4de241291da499bac2d097e93326414 P f6108757b23a46dfb58cf5e7350f3e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4750e9a921794581ae20ed0f1e6d7b4a" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 33795 } } peers { permanent_uuid: "b503ed7b0f3a4950b37c97e2b6f96b9e" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 43569 } } peers { permanent_uuid: "f6108757b23a46dfb58cf5e7350f3e85" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41815 } }
I20260808 22:34:05.452997 20452 ts_tablet_manager.cc:1428] T f4de241291da499bac2d097e93326414 P f6108757b23a46dfb58cf5e7350f3e85: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 22:34:05.453393 20226 heartbeater.cc:503] Master 127.31.87.190:45947 was elected leader, sending a full tablet report...
I20260808 22:34:05.458659 20453 tablet_bootstrap.cc:492] T f4de241291da499bac2d097e93326414 P b503ed7b0f3a4950b37c97e2b6f96b9e: No bootstrap required, opened a new log
I20260808 22:34:05.458860 20453 ts_tablet_manager.cc:1397] T f4de241291da499bac2d097e93326414 P b503ed7b0f3a4950b37c97e2b6f96b9e: Time spent bootstrapping tablet: real 0.025s user 0.005s sys 0.000s
I20260808 22:34:05.463284 20453 raft_consensus.cc:348] T f4de241291da499bac2d097e93326414 P b503ed7b0f3a4950b37c97e2b6f96b9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4750e9a921794581ae20ed0f1e6d7b4a" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 33795 } } peers { permanent_uuid: "b503ed7b0f3a4950b37c97e2b6f96b9e" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 43569 } } peers { permanent_uuid: "f6108757b23a46dfb58cf5e7350f3e85" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41815 } }
I20260808 22:34:05.463578 20453 raft_consensus.cc:374] T f4de241291da499bac2d097e93326414 P b503ed7b0f3a4950b37c97e2b6f96b9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:34:05.463658 20453 raft_consensus.cc:729] T f4de241291da499bac2d097e93326414 P b503ed7b0f3a4950b37c97e2b6f96b9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b503ed7b0f3a4950b37c97e2b6f96b9e, State: Initialized, Role: FOLLOWER
I20260808 22:34:05.464123 20453 consensus_queue.cc:260] T f4de241291da499bac2d097e93326414 P b503ed7b0f3a4950b37c97e2b6f96b9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4750e9a921794581ae20ed0f1e6d7b4a" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 33795 } } peers { permanent_uuid: "b503ed7b0f3a4950b37c97e2b6f96b9e" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 43569 } } peers { permanent_uuid: "f6108757b23a46dfb58cf5e7350f3e85" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41815 } }
I20260808 22:34:05.465170 20453 ts_tablet_manager.cc:1428] T f4de241291da499bac2d097e93326414 P b503ed7b0f3a4950b37c97e2b6f96b9e: Time spent starting tablet: real 0.006s user 0.001s sys 0.005s
I20260808 22:34:05.465549 20419 heartbeater.cc:503] Master 127.31.87.190:45947 was elected leader, sending a full tablet report...
I20260808 22:34:05.467494 20459 raft_consensus.cc:482] T f4de241291da499bac2d097e93326414 P 4750e9a921794581ae20ed0f1e6d7b4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:34:05.467669 20459 raft_consensus.cc:504] T f4de241291da499bac2d097e93326414 P 4750e9a921794581ae20ed0f1e6d7b4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4750e9a921794581ae20ed0f1e6d7b4a" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 33795 } } peers { permanent_uuid: "b503ed7b0f3a4950b37c97e2b6f96b9e" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 43569 } } peers { permanent_uuid: "f6108757b23a46dfb58cf5e7350f3e85" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41815 } }
I20260808 22:34:05.468830 20459 leader_election.cc:288] T f4de241291da499bac2d097e93326414 P 4750e9a921794581ae20ed0f1e6d7b4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b503ed7b0f3a4950b37c97e2b6f96b9e (127.31.87.131:43569), f6108757b23a46dfb58cf5e7350f3e85 (127.31.87.130:41815)
I20260808 22:34:05.477387 20448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:05.469939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:51865 (local address 127.31.87.130:41815)
0808 22:34:05.470035 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:05.470039 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:05.470648 (+ 609us) server_negotiation.cc:407] Connection header received
0808 22:34:05.470837 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:05.470841 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:05.470887 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:05.470933 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:34:05.471342 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:05.472143 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:05.474558 (+ 2415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:05.475113 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:05.475202 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:05.475253 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:05.477188 (+ 1935us) server_negotiation.cc:299] Negotiation successful
0808 22:34:05.477267 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 22:34:05.477937 20462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:05.469614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.130:41815 (local address 127.31.87.129:51865)
0808 22:34:05.470510 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:05.470524 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:34:05.470726 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:34:05.471066 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:34:05.471069 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:34:05.471087 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:34:05.471222 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:05.471228 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:05.472368 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:05.472372 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:05.473628 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:05.473640 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:05.475348 (+ 1708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:05.475351 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:05.475445 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:05.475466 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 22:34:05.477714 (+ 2248us) client_negotiation.cc:240] Negotiation successful
0808 22:34:05.477788 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":832,"thread_start_us":99,"threads_started":1}
I20260808 22:34:05.478667 20159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4de241291da499bac2d097e93326414" candidate_uuid: "4750e9a921794581ae20ed0f1e6d7b4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6108757b23a46dfb58cf5e7350f3e85" is_pre_election: true
I20260808 22:34:05.478995 20159 raft_consensus.cc:2454] T f4de241291da499bac2d097e93326414 P f6108757b23a46dfb58cf5e7350f3e85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4750e9a921794581ae20ed0f1e6d7b4a in term 0.
I20260808 22:34:05.479590 19883 leader_election.cc:302] T f4de241291da499bac2d097e93326414 P 4750e9a921794581ae20ed0f1e6d7b4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4750e9a921794581ae20ed0f1e6d7b4a, f6108757b23a46dfb58cf5e7350f3e85; no voters:
I20260808 22:34:05.479878 20459 raft_consensus.cc:2781] T f4de241291da499bac2d097e93326414 P 4750e9a921794581ae20ed0f1e6d7b4a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:34:05.479975 20459 raft_consensus.cc:482] T f4de241291da499bac2d097e93326414 P 4750e9a921794581ae20ed0f1e6d7b4a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:34:05.480016 20459 raft_consensus.cc:3037] T f4de241291da499bac2d097e93326414 P 4750e9a921794581ae20ed0f1e6d7b4a [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:34:05.481628 20459 raft_consensus.cc:504] T f4de241291da499bac2d097e93326414 P 4750e9a921794581ae20ed0f1e6d7b4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4750e9a921794581ae20ed0f1e6d7b4a" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 33795 } } peers { permanent_uuid: "b503ed7b0f3a4950b37c97e2b6f96b9e" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 43569 } } peers { permanent_uuid: "f6108757b23a46dfb58cf5e7350f3e85" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41815 } }
I20260808 22:34:05.481980 20459 leader_election.cc:288] T f4de241291da499bac2d097e93326414 P 4750e9a921794581ae20ed0f1e6d7b4a [CANDIDATE]: Term 1 election: Requested vote from peers b503ed7b0f3a4950b37c97e2b6f96b9e (127.31.87.131:43569), f6108757b23a46dfb58cf5e7350f3e85 (127.31.87.130:41815)
I20260808 22:34:05.482497 20159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4de241291da499bac2d097e93326414" candidate_uuid: "4750e9a921794581ae20ed0f1e6d7b4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6108757b23a46dfb58cf5e7350f3e85"
I20260808 22:34:05.482622 20159 raft_consensus.cc:3037] T f4de241291da499bac2d097e93326414 P f6108757b23a46dfb58cf5e7350f3e85 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:34:05.483883 20159 raft_consensus.cc:2454] T f4de241291da499bac2d097e93326414 P f6108757b23a46dfb58cf5e7350f3e85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4750e9a921794581ae20ed0f1e6d7b4a in term 1.
I20260808 22:34:05.484334 19883 leader_election.cc:302] T f4de241291da499bac2d097e93326414 P 4750e9a921794581ae20ed0f1e6d7b4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4750e9a921794581ae20ed0f1e6d7b4a, f6108757b23a46dfb58cf5e7350f3e85; no voters:
I20260808 22:34:05.484546 20459 raft_consensus.cc:2781] T f4de241291da499bac2d097e93326414 P 4750e9a921794581ae20ed0f1e6d7b4a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:34:05.484762 20459 raft_consensus.cc:686] T f4de241291da499bac2d097e93326414 P 4750e9a921794581ae20ed0f1e6d7b4a [term 1 LEADER]: Becoming Leader. State: Replica: 4750e9a921794581ae20ed0f1e6d7b4a, State: Running, Role: LEADER
I20260808 22:34:05.485116 20459 consensus_queue.cc:237] T f4de241291da499bac2d097e93326414 P 4750e9a921794581ae20ed0f1e6d7b4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4750e9a921794581ae20ed0f1e6d7b4a" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 33795 } } peers { permanent_uuid: "b503ed7b0f3a4950b37c97e2b6f96b9e" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 43569 } } peers { permanent_uuid: "f6108757b23a46dfb58cf5e7350f3e85" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41815 } }
I20260808 22:34:05.488299 19756 catalog_manager.cc:5495] T f4de241291da499bac2d097e93326414 P 4750e9a921794581ae20ed0f1e6d7b4a reported cstate change: term changed from 0 to 1, leader changed from <none> to 4750e9a921794581ae20ed0f1e6d7b4a (127.31.87.129). New cstate: current_term: 1 leader_uuid: "4750e9a921794581ae20ed0f1e6d7b4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4750e9a921794581ae20ed0f1e6d7b4a" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 33795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b503ed7b0f3a4950b37c97e2b6f96b9e" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 43569 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6108757b23a46dfb58cf5e7350f3e85" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41815 } health_report { overall_health: UNKNOWN } } }
I20260808 22:34:05.492094 20450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:05.469230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:47783 (local address 127.31.87.131:43569)
0808 22:34:05.469333 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:05.469337 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:05.473945 (+ 4608us) server_negotiation.cc:407] Connection header received
0808 22:34:05.474122 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:05.474125 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:05.474166 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:05.474208 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:34:05.475587 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:05.476278 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:05.479089 (+ 2811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:05.479626 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:05.479714 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:05.479763 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:05.491855 (+ 12092us) server_negotiation.cc:299] Negotiation successful
0808 22:34:05.491954 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 22:34:05.492712 20463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:05.470172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.131:43569 (local address 127.31.87.129:47783)
0808 22:34:05.473872 (+ 3700us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:05.473885 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:34:05.474006 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:34:05.474323 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:34:05.474325 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:34:05.474341 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:34:05.474439 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:05.474444 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:05.476433 (+ 1989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:05.476436 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:05.478951 (+ 2515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:05.478962 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:05.491545 (+ 12583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:05.491548 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:05.491692 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:05.491719 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 22:34:05.492452 (+ 733us) client_negotiation.cc:240] Negotiation successful
0808 22:34:05.492544 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3632,"thread_start_us":101,"threads_started":1}
I20260808 22:34:05.493614 20356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4de241291da499bac2d097e93326414" candidate_uuid: "4750e9a921794581ae20ed0f1e6d7b4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b503ed7b0f3a4950b37c97e2b6f96b9e" is_pre_election: true
I20260808 22:34:05.493891 20356 raft_consensus.cc:2454] T f4de241291da499bac2d097e93326414 P b503ed7b0f3a4950b37c97e2b6f96b9e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4750e9a921794581ae20ed0f1e6d7b4a in term 0.
I20260808 22:34:05.494204 20355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4de241291da499bac2d097e93326414" candidate_uuid: "4750e9a921794581ae20ed0f1e6d7b4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b503ed7b0f3a4950b37c97e2b6f96b9e"
I20260808 22:34:05.494302 20355 raft_consensus.cc:3037] T f4de241291da499bac2d097e93326414 P b503ed7b0f3a4950b37c97e2b6f96b9e [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:34:05.495530 20355 raft_consensus.cc:2454] T f4de241291da499bac2d097e93326414 P b503ed7b0f3a4950b37c97e2b6f96b9e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4750e9a921794581ae20ed0f1e6d7b4a in term 1.
I20260808 22:34:05.514909 20446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:05.510130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52770 (local address 127.31.87.129:33795)
0808 22:34:05.510242 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:05.510246 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:05.510258 (+ 12us) server_negotiation.cc:407] Connection header received
0808 22:34:05.510330 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:05.510333 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:05.510391 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:05.510488 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:34:05.510865 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:05.511625 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:05.512823 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:05.513186 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:05.513290 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:05.513712 (+ 422us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:34:05.514214 (+ 502us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:34:05.514323 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:05.514711 (+ 388us) server_negotiation.cc:299] Negotiation successful
0808 22:34:05.514765 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 22:34:05.522238 20159 raft_consensus.cc:1261] T f4de241291da499bac2d097e93326414 P f6108757b23a46dfb58cf5e7350f3e85 [term 1 FOLLOWER]: Refusing update from remote peer 4750e9a921794581ae20ed0f1e6d7b4a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:34:05.523131 20464 consensus_queue.cc:1035] T f4de241291da499bac2d097e93326414 P 4750e9a921794581ae20ed0f1e6d7b4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6108757b23a46dfb58cf5e7350f3e85" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:34:05.529879 20355 raft_consensus.cc:1261] T f4de241291da499bac2d097e93326414 P b503ed7b0f3a4950b37c97e2b6f96b9e [term 1 FOLLOWER]: Refusing update from remote peer 4750e9a921794581ae20ed0f1e6d7b4a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:34:05.530583 20464 consensus_queue.cc:1035] T f4de241291da499bac2d097e93326414 P 4750e9a921794581ae20ed0f1e6d7b4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b503ed7b0f3a4950b37c97e2b6f96b9e" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 43569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:34:05.545210 20470 mvcc.cc:205] Tried to move back new op lower bound from 7316391712849993728 to 7316391712710631424. Current Snapshot: MvccSnapshot[applied={T|T < 7316391712849993728}]
I20260808 22:34:05.548281 20473 mvcc.cc:205] Tried to move back new op lower bound from 7316391712849993728 to 7316391712710631424. Current Snapshot: MvccSnapshot[applied={T|T < 7316391712849993728 or (T in {7316391712849993728})}]
I20260808 22:34:05.558463 19756 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44956:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:34:05.558632 19756 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44956:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:34:05.561587 19756 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ddb477d3689f4bbb812a0b93b43c1673: Sending DeleteTablet for 3 replicas of tablet f4de241291da499bac2d097e93326414
I20260808 22:34:05.562644 20105 tablet_service.cc:1514] Processing DeleteTablet for tablet f4de241291da499bac2d097e93326414 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:34:05 UTC) from {username='kudu', principal='kudu/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:56870
I20260808 22:34:05.563573 20331 tablet_service.cc:1514] Processing DeleteTablet for tablet f4de241291da499bac2d097e93326414 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:34:05 UTC) from {username='kudu', principal='kudu/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:40268
I20260808 22:34:05.564210 19940 tablet_service.cc:1514] Processing DeleteTablet for tablet f4de241291da499bac2d097e93326414 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:34:05 UTC) from {username='kudu', principal='kudu/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:52764
I20260808 22:34:05.565527 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 19808
I20260808 22:34:05.576781 20484 tablet_replica.cc:330] T f4de241291da499bac2d097e93326414 P b503ed7b0f3a4950b37c97e2b6f96b9e: stopping tablet replica
I20260808 22:34:05.577158 20484 raft_consensus.cc:2229] T f4de241291da499bac2d097e93326414 P b503ed7b0f3a4950b37c97e2b6f96b9e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:34:05.577512 20484 raft_consensus.cc:2258] T f4de241291da499bac2d097e93326414 P b503ed7b0f3a4950b37c97e2b6f96b9e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:34:05.582677 20483 tablet_replica.cc:330] T f4de241291da499bac2d097e93326414 P f6108757b23a46dfb58cf5e7350f3e85: stopping tablet replica
I20260808 22:34:05.582981 20483 raft_consensus.cc:2229] T f4de241291da499bac2d097e93326414 P f6108757b23a46dfb58cf5e7350f3e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:34:05.583279 20483 raft_consensus.cc:2258] T f4de241291da499bac2d097e93326414 P f6108757b23a46dfb58cf5e7350f3e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:34:05.584944 20484 ts_tablet_manager.cc:1905] T f4de241291da499bac2d097e93326414 P b503ed7b0f3a4950b37c97e2b6f96b9e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:34:05.585911 20483 ts_tablet_manager.cc:1905] T f4de241291da499bac2d097e93326414 P f6108757b23a46dfb58cf5e7350f3e85: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:34:05.588657 20483 ts_tablet_manager.cc:1918] T f4de241291da499bac2d097e93326414 P f6108757b23a46dfb58cf5e7350f3e85: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:34:05.591245 20484 ts_tablet_manager.cc:1918] T f4de241291da499bac2d097e93326414 P b503ed7b0f3a4950b37c97e2b6f96b9e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:34:05.594126 20483 log.cc:1196] T f4de241291da499bac2d097e93326414 P f6108757b23a46dfb58cf5e7350f3e85: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-1/wal/wals/f4de241291da499bac2d097e93326414
I20260808 22:34:05.594251 20484 log.cc:1196] T f4de241291da499bac2d097e93326414 P b503ed7b0f3a4950b37c97e2b6f96b9e: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786228291013016-32094-0/minicluster-data/ts-2/wal/wals/f4de241291da499bac2d097e93326414
I20260808 22:34:05.594696 20484 ts_tablet_manager.cc:1939] T f4de241291da499bac2d097e93326414 P b503ed7b0f3a4950b37c97e2b6f96b9e: Deleting consensus metadata
W20260808 22:34:05.594990 19736 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.87.129:33795 (error 108)
I20260808 22:34:05.595072 20483 ts_tablet_manager.cc:1939] T f4de241291da499bac2d097e93326414 P f6108757b23a46dfb58cf5e7350f3e85: Deleting consensus metadata
I20260808 22:34:05.597340 19739 catalog_manager.cc:4841] TS f6108757b23a46dfb58cf5e7350f3e85 (127.31.87.130:41815): tablet f4de241291da499bac2d097e93326414 (table test-table [id=08d93ced49944b67a14638225f91cbe7]) successfully deleted
I20260808 22:34:05.597568 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 20046
Aug 08 22:34:05 dist-test-slave-266d krb5kdc[18310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228444, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for HTTP/127.31.87.148@KRBTEST.COM
I20260808 22:34:05.603541 19736 catalog_manager.cc:4841] TS b503ed7b0f3a4950b37c97e2b6f96b9e (127.31.87.131:43569): tablet f4de241291da499bac2d097e93326414 (table test-table [id=08d93ced49944b67a14638225f91cbe7]) successfully deleted
I20260808 22:34:05.624564 20447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:05.624195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:33795 (local address 127.0.0.1:52786)
0808 22:34:05.624375 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:05.624464 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.129:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260808 22:34:05.624764 19736 catalog_manager.cc:4584] TS 4750e9a921794581ae20ed0f1e6d7b4a (127.31.87.129:33795): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f4de241291da499bac2d097e93326414: Network error: Client connection negotiation failed: client connection to 127.31.87.129:33795: connect: Connection refused (error 111)
I20260808 22:34:05.625600 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 20231
I20260808 22:34:05.649786 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 19705
I20260808 22:34:05.663228 32094 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 22:34:06.009809 32094 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 22:34:06.009922 32094 mini_ranger.cc:66] Stopping Ranger...
I20260808 22:34:06.353608 32094 mini_ranger.cc:68] Stopped Ranger
2026-08-08 22:34:06.353 UTC [18355] LOG: received smart shutdown request
2026-08-08 22:34:06.357 UTC [18355] LOG: background worker "logical replication launcher" (PID 18369) exited with exit code 1
2026-08-08 22:34:06.358 UTC [18360] LOG: shutting down
2026-08-08 22:34:06.492 UTC [18355] LOG: database system is shut down
2026-08-08T22:34:06Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (47435 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:34:06 dist-test-slave-266d krb5kdc[20503](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:34:06 dist-test-slave-266d krb5kdc[20503](info): set up 2 sockets
Aug 08 22:34:06 dist-test-slave-266d krb5kdc[20503](info): commencing operation
krb5kdc: starting...
W20260808 22:34:07.861927 32094 mini_kdc.cc:121] Time spent starting KDC: real 1.254s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Aug 08 22:34:07 dist-test-slave-266d krb5kdc[20503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228447, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:34:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:34:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.87.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
I20260808 22:34:07.977123 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:45141
--webserver_interface=127.31.87.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:34445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:45141,127.31.87.189:43003,127.31.87.188:46191
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:34:08.082162 20527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:34:08.082386 20527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:34:08.082423 20527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:34:08.086247 20527 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:34:08.086290 20527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:34:08.086309 20527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:34:08.086338 20527 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:34:08.086365 20527 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:34:08.091243 20527 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:34445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:45141,127.31.87.189:43003,127.31.87.188:46191
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:45141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.87.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20527
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:34:08.092103 20527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:34:08.093011 20527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:34:08.100713 20535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:34:08.100787 20539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:34:08.100731 20536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:34:08.100955 20527 server_base.cc:1034] running on GCE node
I20260808 22:34:08.101564 20527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:34:08.102531 20527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:34:08.103677 20527 hybrid_clock.cc:648] HybridClock initialized: now 1786228448103655 us; error 36 us; skew 500 ppm
Aug 08 22:34:08 dist-test-slave-266d krb5kdc[20503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228448, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:34:08.106159 20527 init.cc:377] Logged in from keytab as kudu/127.31.87.190@KRBTEST.COM (short username kudu)
I20260808 22:34:08.106958 20527 webserver.cc:460] Webserver started at http://127.31.87.190:41121/ using document root <none> and password file <none>
I20260808 22:34:08.107447 20527 fs_manager.cc:359] Metadata directory not provided
I20260808 22:34:08.107508 20527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:34:08.107684 20527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:34:08.109589 20527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/minicluster-data/master-0/data/instance:
uuid: "22619f565175445b8e75aa31f358ff31"
format_stamp: "Formatted at 2026-08-08 22:34:08 on dist-test-slave-266d"
server_key: "22cc8769df365ae08e4bd9d4591b5a0c"
server_key_iv: "64eb7a7823ea24f62c38a7ab2b246531"
server_key_version: "encryptionkey@0"
I20260808 22:34:08.110098 20527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/minicluster-data/master-0/wal/instance:
uuid: "22619f565175445b8e75aa31f358ff31"
format_stamp: "Formatted at 2026-08-08 22:34:08 on dist-test-slave-266d"
server_key: "22cc8769df365ae08e4bd9d4591b5a0c"
server_key_iv: "64eb7a7823ea24f62c38a7ab2b246531"
server_key_version: "encryptionkey@0"
I20260808 22:34:08.113467 20527 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:34:08.115631 20553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:08.116700 20527 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:34:08.116837 20527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "22619f565175445b8e75aa31f358ff31"
format_stamp: "Formatted at 2026-08-08 22:34:08 on dist-test-slave-266d"
server_key: "22cc8769df365ae08e4bd9d4591b5a0c"
server_key_iv: "64eb7a7823ea24f62c38a7ab2b246531"
server_key_version: "encryptionkey@0"
I20260808 22:34:08.116922 20527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:34:08.137982 20527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:34:08.140511 20527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:34:08.140668 20527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:34:08.147018 20527 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.190:45141
I20260808 22:34:08.147033 20625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.190:45141 every 8 connection(s)
I20260808 22:34:08.147913 20527 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:34:08.150533 20628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:34:08.152244 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 20527
I20260808 22:34:08.152376 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/minicluster-data/master-0/wal/instance
I20260808 22:34:08.152617 32094 external_mini_cluster.cc:1428] Setting key 08e6ad43f51c70caa461f3fe73317026
I20260808 22:34:08.154212 20628 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45141 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.31.87.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.189@KRBTEST.COM" created.
Aug 08 22:34:08 dist-test-slave-266d krb5kdc[20503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228448, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:34:08.168459 20632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:08.156305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45141 (local address 127.0.0.1:39078)
0808 22:34:08.156822 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:08.156857 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 22:34:08.157655 (+ 798us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:34:08.158289 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:34:08.158294 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:34:08.158707 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:34:08.158928 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:08.158941 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:08.160025 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:08.160028 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:08.160760 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:08.160770 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:08.161404 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:08.161408 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:08.161530 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:08.162052 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:34:08.162067 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:34:08.163790 (+ 1723us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:34:08.165835 (+ 2045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:08.165843 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:08.165852 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:08.166070 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:08.166359 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:08.166362 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:08.166364 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:08.166465 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:08.166981 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:34:08.166987 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:34:08.167114 (+ 127us) client_negotiation.cc:769] Sending connection context
0808 22:34:08.167254 (+ 140us) client_negotiation.cc:240] Negotiation successful
0808 22:34:08.167586 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"mutex_wait_us":87,"spinlock_wait_cycles":5888,"thread_start_us":150,"threads_started":1}
I20260808 22:34:08.168459 20631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:08.156305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39078 (local address 127.31.87.190:45141)
0808 22:34:08.156686 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:08.156703 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:08.156953 (+ 250us) server_negotiation.cc:407] Connection header received
0808 22:34:08.157941 (+ 988us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:08.157945 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:08.158120 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:08.158273 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:34:08.159072 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:08.159912 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:08.160868 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:08.161251 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:08.161391 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:08.163919 (+ 2528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:34:08.163925 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:34:08.163934 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:34:08.163954 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:34:08.165732 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:08.166172 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:08.166177 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:08.166178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:08.166276 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:08.166545 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:08.166548 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:08.166550 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:08.166877 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:34:08.166991 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:08.167329 (+ 338us) server_negotiation.cc:299] Negotiation successful
0808 22:34:08.167520 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":240,"thread_start_us":103,"threads_started":1}
I20260808 22:34:08.170800 20628 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.87.189" port: 43003 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:34:08.171399 20632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:08.171184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.189:43003 (local address 127.0.0.1:50946)
0808 22:34:08.171276 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:08.171339 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.189:43003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 22:34:08.171602 20559 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.87.189:43003: connect: Connection refused (error 111)
I20260808 22:34:08.172346 20632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:08.172148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.189:43003 (local address 127.0.0.1:50954)
0808 22:34:08.172244 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:08.172292 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.189:43003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 22:34:08.172465 20628 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.87.189:43003: Network error: Client connection negotiation failed: client connection to 127.31.87.189:43003: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.189.keytab.
Entry for principal kudu/127.31.87.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.189.keytab.
I20260808 22:34:08.177335 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.189:43003
--webserver_interface=127.31.87.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.189.keytab
--principal=kudu/127.31.87.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:34445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:45141,127.31.87.189:43003,127.31.87.188:46191
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 22:34:08.221657 20628 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.87.189" port: 43003 } attempt: 1
I20260808 22:34:08.222497 20632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:08.222153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.189:43003 (local address 127.0.0.1:50968)
0808 22:34:08.222324 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:08.222400 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.189:43003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260808 22:34:08.223307 20632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:08.223073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.189:43003 (local address 127.0.0.1:50972)
0808 22:34:08.223182 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:08.223238 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.189:43003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 22:34:08.223462 20628 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.87.189:43003: Network error: Client connection negotiation failed: client connection to 127.31.87.189:43003: connect: Connection refused (error 111)
W20260808 22:34:08.282791 20637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:34:08.283002 20637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:34:08.283038 20637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:34:08.286815 20637 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:34:08.286854 20637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:34:08.286871 20637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:34:08.286886 20637 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:34:08.286899 20637 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:34:08.291641 20628 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.87.189" port: 43003 } attempt: 2
I20260808 22:34:08.291716 20637 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:34445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:45141,127.31.87.189:43003,127.31.87.188:46191
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.189.keytab
--principal=kudu/127.31.87.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.189:43003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.31.87.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20637
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:34:08.292491 20632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:08.292155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.189:43003 (local address 127.0.0.1:50976)
0808 22:34:08.292329 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:08.292402 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.189:43003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260808 22:34:08.292687 20637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:34:08.293237 20632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:08.292976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.189:43003 (local address 127.0.0.1:50980)
0808 22:34:08.293115 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:08.293170 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.189:43003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 22:34:08.293392 20628 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.87.189:43003: Network error: Client connection negotiation failed: client connection to 127.31.87.189:43003: connect: Connection refused (error 111)
I20260808 22:34:08.293713 20637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:34:08.301384 20645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:34:08.301522 20637 server_base.cc:1034] running on GCE node
W20260808 22:34:08.301412 20646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:34:08.301456 20648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:34:08.302155 20637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:34:08.303128 20637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:34:08.304303 20637 hybrid_clock.cc:648] HybridClock initialized: now 1786228448304293 us; error 31 us; skew 500 ppm
Aug 08 22:34:08 dist-test-slave-266d krb5kdc[20503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228448, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:34:08.306886 20637 init.cc:377] Logged in from keytab as kudu/127.31.87.189@KRBTEST.COM (short username kudu)
I20260808 22:34:08.307725 20637 webserver.cc:460] Webserver started at http://127.31.87.189:42163/ using document root <none> and password file <none>
I20260808 22:34:08.308256 20637 fs_manager.cc:359] Metadata directory not provided
I20260808 22:34:08.308328 20637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:34:08.308537 20637 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:34:08.310496 20637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/minicluster-data/master-1/data/instance:
uuid: "c8dadf19214245e4806db1f1b1639dab"
format_stamp: "Formatted at 2026-08-08 22:34:08 on dist-test-slave-266d"
server_key: "7d2943813c7bb0f708d4fb5a1b5f9207"
server_key_iv: "ebb83141b3eaa799b1225945590513f6"
server_key_version: "encryptionkey@0"
I20260808 22:34:08.311028 20637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/minicluster-data/master-1/wal/instance:
uuid: "c8dadf19214245e4806db1f1b1639dab"
format_stamp: "Formatted at 2026-08-08 22:34:08 on dist-test-slave-266d"
server_key: "7d2943813c7bb0f708d4fb5a1b5f9207"
server_key_iv: "ebb83141b3eaa799b1225945590513f6"
server_key_version: "encryptionkey@0"
I20260808 22:34:08.314524 20637 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 22:34:08.316704 20656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:08.317832 20637 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:34:08.317960 20637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/minicluster-data/master-1/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/minicluster-data/master-1/wal
uuid: "c8dadf19214245e4806db1f1b1639dab"
format_stamp: "Formatted at 2026-08-08 22:34:08 on dist-test-slave-266d"
server_key: "7d2943813c7bb0f708d4fb5a1b5f9207"
server_key_iv: "ebb83141b3eaa799b1225945590513f6"
server_key_version: "encryptionkey@0"
I20260808 22:34:08.318053 20637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:34:08.367071 20637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:34:08.369993 20637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:34:08.370201 20637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:34:08.377283 20637 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.189:43003
I20260808 22:34:08.377329 20727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.189:43003 every 8 connection(s)
I20260808 22:34:08.378257 20637 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/minicluster-data/master-1/data/info.pb
I20260808 22:34:08.380869 20728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:34:08.382997 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 20637
I20260808 22:34:08.383141 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/minicluster-data/master-1/wal/instance
I20260808 22:34:08.383435 32094 external_mini_cluster.cc:1428] Setting key 570369ab16519add22fed1703175b82d
I20260808 22:34:08.384550 20628 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.87.189" port: 43003 } attempt: 3
I20260808 22:34:08.385161 20728 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45141 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 22:34:08 dist-test-slave-266d krb5kdc[20503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228448, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.189@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.87.188@KRBTEST.COM; defaulting to no policy
Aug 08 22:34:08 dist-test-slave-266d krb5kdc[20503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228448, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for kudu/127.31.87.189@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.188@KRBTEST.COM" created.
I20260808 22:34:08.397697 20632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:08.385114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.189:43003 (local address 127.0.0.1:50988)
0808 22:34:08.385345 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:08.385363 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 22:34:08.385482 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:34:08.387840 (+ 2358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:34:08.387843 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:34:08.388049 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:34:08.388211 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:08.388220 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:08.389957 (+ 1737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:08.389961 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:08.390783 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:08.390796 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:08.391506 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:08.391510 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:08.391613 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:08.392120 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:34:08.392139 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:34:08.393754 (+ 1615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:34:08.395704 (+ 1950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:08.395707 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:08.395709 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:08.396010 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:08.396324 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:08.396328 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:08.396331 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:08.396419 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:08.397075 (+ 656us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:34:08.397077 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:34:08.397213 (+ 136us) client_negotiation.cc:769] Sending connection context
0808 22:34:08.397332 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 22:34:08.397471 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"spinlock_wait_cycles":1792}
I20260808 22:34:08.398546 20731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:08.385877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50988 (local address 127.31.87.189:43003)
0808 22:34:08.386362 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:08.386379 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:08.386417 (+ 38us) server_negotiation.cc:407] Connection header received
0808 22:34:08.387130 (+ 713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:08.387155 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:08.387571 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:08.387861 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:34:08.388372 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:08.389827 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:08.390906 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:08.391401 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:08.391492 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:08.393892 (+ 2400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:34:08.393897 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:34:08.393907 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:34:08.393929 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:34:08.395616 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:08.396106 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:08.396112 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:08.396114 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:08.396187 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:08.396506 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:08.396509 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:08.396511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:08.396964 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:34:08.397113 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:08.397496 (+ 383us) server_negotiation.cc:299] Negotiation successful
0808 22:34:08.397790 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":337,"thread_start_us":163,"threads_started":1}
I20260808 22:34:08.398830 20631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:08.387077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39080 (local address 127.31.87.190:45141)
0808 22:34:08.387207 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:08.387211 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:08.387577 (+ 366us) server_negotiation.cc:407] Connection header received
0808 22:34:08.387781 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:08.387785 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:08.387835 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:08.387883 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:34:08.388870 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:08.389660 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:08.390675 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:08.390935 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:08.391034 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:08.393658 (+ 2624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:34:08.393662 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:34:08.393665 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:34:08.393692 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:34:08.395089 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:08.395704 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:08.395707 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:08.395709 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:08.395855 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:08.397167 (+ 1312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:08.397170 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:08.397172 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:08.397373 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:34:08.397467 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:08.398552 (+ 1085us) server_negotiation.cc:299] Negotiation successful
0808 22:34:08.398670 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62,"spinlock_wait_cycles":53632}
I20260808 22:34:08.399277 20732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:08.387041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45141 (local address 127.0.0.1:39080)
0808 22:34:08.387445 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:08.387491 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 22:34:08.387653 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:34:08.388061 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:34:08.388067 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:34:08.388523 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:34:08.388714 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:08.388726 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:08.389792 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:08.389795 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:08.390564 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:08.390573 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:08.391116 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:08.391121 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:08.391363 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:08.391993 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:34:08.392014 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:34:08.393519 (+ 1505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:34:08.395198 (+ 1679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:08.395207 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:08.395216 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:08.395602 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:08.395968 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:08.395972 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:08.395974 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:08.396168 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:08.397526 (+ 1358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:34:08.397533 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:34:08.397660 (+ 127us) client_negotiation.cc:769] Sending connection context
0808 22:34:08.398965 (+ 1305us) client_negotiation.cc:240] Negotiation successful
0808 22:34:08.399143 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"mutex_wait_us":4,"thread_start_us":95,"threads_started":1}
I20260808 22:34:08.400424 20728 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.87.189" port: 43003 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:34:08.400491 20628 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.87.188" port: 46191 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:34:08.401196 20632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:08.400955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.188:46191 (local address 127.0.0.1:47082)
0808 22:34:08.401049 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:08.401129 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.188:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 22:34:08.401978 20632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:08.401749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.188:46191 (local address 127.0.0.1:47084)
0808 22:34:08.401880 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:08.401926 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.188:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260808 22:34:08.402128 20628 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.87.188:46191: Network error: Client connection negotiation failed: client connection to 127.31.87.188:46191: connect: Connection refused (error 111)
Aug 08 22:34:08 dist-test-slave-266d krb5kdc[20503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228448, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.189@KRBTEST.COM for kudu/127.31.87.189@KRBTEST.COM
I20260808 22:34:08.407994 20732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:08.400923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.189:43003 (local address 127.0.0.1:50998)
0808 22:34:08.401111 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:08.401126 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:34:08.401203 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:34:08.401420 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:34:08.401423 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:34:08.401560 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:34:08.401638 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:08.401645 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:08.402616 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:08.402619 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:08.403126 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:08.403134 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:08.403705 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:08.403707 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:08.403791 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:08.404176 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:34:08.404188 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:34:08.405357 (+ 1169us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:34:08.406850 (+ 1493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:08.406853 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:08.406855 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:08.407044 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:08.407259 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:08.407262 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:08.407264 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:08.407355 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:08.407643 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:34:08.407645 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:34:08.407717 (+ 72us) client_negotiation.cc:769] Sending connection context
0808 22:34:08.407794 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 22:34:08.407887 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"mutex_wait_us":1}
I20260808 22:34:08.408125 20731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:08.401144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50998 (local address 127.31.87.189:43003)
0808 22:34:08.401236 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:08.401239 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:08.401255 (+ 16us) server_negotiation.cc:407] Connection header received
0808 22:34:08.401297 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:08.401299 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:08.401337 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:08.401409 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:34:08.401757 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:08.402507 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:08.403241 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:08.403592 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:08.403752 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:08.405483 (+ 1731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:34:08.405486 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:34:08.405488 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:34:08.405508 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:34:08.406740 (+ 1232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:08.407132 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:08.407134 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:08.407135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:08.407190 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:08.407429 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:08.407431 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:08.407432 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:08.407574 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:34:08.407642 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:08.407854 (+ 212us) server_negotiation.cc:299] Negotiation successful
0808 22:34:08.407972 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":37}
I20260808 22:34:08.408771 20728 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.87.188" port: 46191 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:34:08.409341 20732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:08.409129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.188:46191 (local address 127.0.0.1:47090)
0808 22:34:08.409219 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:08.409290 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.188:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260808 22:34:08.409935 20660 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.87.188:46191: connect: Connection refused (error 111)
I20260808 22:34:08.410737 20732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:08.410558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.188:46191 (local address 127.0.0.1:47102)
0808 22:34:08.410639 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:08.410686 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.188:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260808 22:34:08.410872 20728 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.87.188:46191: Network error: Client connection negotiation failed: client connection to 127.31.87.188:46191: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.188.keytab.
Entry for principal kudu/127.31.87.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.188.keytab.
I20260808 22:34:08.413580 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.188:46191
--webserver_interface=127.31.87.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.188.keytab
--principal=kudu/127.31.87.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:34445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:45141,127.31.87.189:43003,127.31.87.188:46191
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 22:34:08.433260 20628 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.87.188" port: 46191 } attempt: 1
I20260808 22:34:08.433954 20632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:08.433700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.188:46191 (local address 127.0.0.1:47112)
0808 22:34:08.433829 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:08.433881 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.188:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 22:34:08.434572 20632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:08.434386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.188:46191 (local address 127.0.0.1:47124)
0808 22:34:08.434482 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:08.434524 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.188:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 22:34:08.434679 20628 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.87.188:46191: Network error: Client connection negotiation failed: client connection to 127.31.87.188:46191: connect: Connection refused (error 111)
I20260808 22:34:08.460104 20728 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.87.188" port: 46191 } attempt: 1
I20260808 22:34:08.460809 20732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:08.460563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.188:46191 (local address 127.0.0.1:47130)
0808 22:34:08.460692 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:08.460743 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.188:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 22:34:08.461495 20732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:08.461290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.188:46191 (local address 127.0.0.1:47136)
0808 22:34:08.461389 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:08.461433 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.188:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 22:34:08.461613 20728 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.87.188:46191: Network error: Client connection negotiation failed: client connection to 127.31.87.188:46191: connect: Connection refused (error 111)
I20260808 22:34:08.509811 20628 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.87.188" port: 46191 } attempt: 2
I20260808 22:34:08.510627 20632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:08.510347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.188:46191 (local address 127.0.0.1:47142)
0808 22:34:08.510490 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:08.510547 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.188:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260808 22:34:08.511271 20632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:08.511070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.188:46191 (local address 127.0.0.1:47156)
0808 22:34:08.511170 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:08.511213 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.188:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 22:34:08.511390 20628 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.87.188:46191: Network error: Client connection negotiation failed: client connection to 127.31.87.188:46191: connect: Connection refused (error 111)
W20260808 22:34:08.522993 20735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:34:08.523212 20735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:34:08.523268 20735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:34:08.527181 20735 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:34:08.527228 20735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:34:08.527254 20735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:34:08.527287 20735 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:34:08.527309 20735 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:34:08.529745 20728 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.87.188" port: 46191 } attempt: 2
I20260808 22:34:08.530368 20732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:08.530140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.188:46191 (local address 127.0.0.1:47168)
0808 22:34:08.530250 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:08.530306 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.188:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 22:34:08.530993 20732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:08.530798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.188:46191 (local address 127.0.0.1:47174)
0808 22:34:08.530900 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:08.530942 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.188:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 22:34:08.531098 20728 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.87.188:46191: Network error: Client connection negotiation failed: client connection to 127.31.87.188:46191: connect: Connection refused (error 111)
I20260808 22:34:08.532580 20735 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:34445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:45141,127.31.87.189:43003,127.31.87.188:46191
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.188.keytab
--principal=kudu/127.31.87.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.188:46191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.31.87.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20735
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:34:08.533526 20735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:34:08.534426 20735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:34:08.542219 20742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:34:08.542265 20749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:34:08.542223 20744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:34:08.542402 20735 server_base.cc:1034] running on GCE node
I20260808 22:34:08.543007 20735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:34:08.543942 20735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:34:08.545109 20735 hybrid_clock.cc:648] HybridClock initialized: now 1786228448545069 us; error 33 us; skew 500 ppm
Aug 08 22:34:08 dist-test-slave-266d krb5kdc[20503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228448, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:34:08.547571 20735 init.cc:377] Logged in from keytab as kudu/127.31.87.188@KRBTEST.COM (short username kudu)
I20260808 22:34:08.548372 20735 webserver.cc:460] Webserver started at http://127.31.87.188:35529/ using document root <none> and password file <none>
I20260808 22:34:08.548859 20735 fs_manager.cc:359] Metadata directory not provided
I20260808 22:34:08.548914 20735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:34:08.549130 20735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:34:08.551007 20735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/minicluster-data/master-2/data/instance:
uuid: "57682fab1aa14cd8bf0bbbddc86da4dc"
format_stamp: "Formatted at 2026-08-08 22:34:08 on dist-test-slave-266d"
server_key: "8223bd5909e3f4f4f006435a7154e259"
server_key_iv: "81a3e84ed3e84a5a28a7661731657802"
server_key_version: "encryptionkey@0"
I20260808 22:34:08.551519 20735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/minicluster-data/master-2/wal/instance:
uuid: "57682fab1aa14cd8bf0bbbddc86da4dc"
format_stamp: "Formatted at 2026-08-08 22:34:08 on dist-test-slave-266d"
server_key: "8223bd5909e3f4f4f006435a7154e259"
server_key_iv: "81a3e84ed3e84a5a28a7661731657802"
server_key_version: "encryptionkey@0"
I20260808 22:34:08.554879 20735 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 22:34:08.556993 20759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:08.558085 20735 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:34:08.558184 20735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/minicluster-data/master-2/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/minicluster-data/master-2/wal
uuid: "57682fab1aa14cd8bf0bbbddc86da4dc"
format_stamp: "Formatted at 2026-08-08 22:34:08 on dist-test-slave-266d"
server_key: "8223bd5909e3f4f4f006435a7154e259"
server_key_iv: "81a3e84ed3e84a5a28a7661731657802"
server_key_version: "encryptionkey@0"
I20260808 22:34:08.558285 20735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:34:08.580754 20735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:34:08.583343 20735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:34:08.583505 20735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:34:08.590010 20735 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.188:46191
I20260808 22:34:08.590029 20826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.188:46191 every 8 connection(s)
I20260808 22:34:08.590867 20735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/minicluster-data/master-2/data/info.pb
I20260808 22:34:08.593472 20827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:34:08.597046 20827 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45141 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:34:08.598968 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 20735
I20260808 22:34:08.599118 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786228291013016-32094-0/minicluster-data/master-2/wal/instance
I20260808 22:34:08.599373 32094 external_mini_cluster.cc:1428] Setting key a809977323c9dededa2c69705b7ec873
Aug 08 22:34:08 dist-test-slave-266d krb5kdc[20503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228448, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.188@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
Aug 08 22:34:08 dist-test-slave-266d krb5kdc[20503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228447, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.87.188@KRBTEST.COM
I20260808 22:34:08.610414 20631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:08.598830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39082 (local address 127.31.87.190:45141)
0808 22:34:08.598975 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:08.598979 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:08.599788 (+ 809us) server_negotiation.cc:407] Connection header received
0808 22:34:08.600721 (+ 933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:08.600724 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:08.600771 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:08.600858 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:34:08.601942 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:08.602759 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:08.604021 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:08.604375 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:08.604495 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:08.606816 (+ 2321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:34:08.606820 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:34:08.606822 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:34:08.606849 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:34:08.608296 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:08.608879 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:08.608881 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:08.608882 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:08.608954 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:08.609293 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:08.609297 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:08.609298 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:08.609495 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:34:08.609594 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:08.610133 (+ 539us) server_negotiation.cc:299] Negotiation successful
0808 22:34:08.610253 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 22:34:08.610586 20628 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.87.188" port: 46191 } attempt: 3
I20260808 22:34:08.611267 20832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:08.600018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47180 (local address 127.31.87.188:46191)
0808 22:34:08.600349 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:08.600361 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:08.600402 (+ 41us) server_negotiation.cc:407] Connection header received
0808 22:34:08.600535 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:08.600549 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:08.600772 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:08.600941 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:34:08.601815 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:08.603066 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:08.604133 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:08.604670 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:08.604756 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:08.607215 (+ 2459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:34:08.607220 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:34:08.607229 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:34:08.607252 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:34:08.608803 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:08.609368 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:08.609372 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:08.609373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:08.609440 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:08.609703 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:08.609706 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:08.609707 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:08.609985 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:34:08.610149 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:08.610378 (+ 229us) server_negotiation.cc:299] Negotiation successful
0808 22:34:08.610845 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"spinlock_wait_cycles":2304,"thread_start_us":73,"threads_started":1}
I20260808 22:34:08.611279 20829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:08.599205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:45141 (local address 127.0.0.1:39082)
0808 22:34:08.599633 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:08.599684 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 22:34:08.600537 (+ 853us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:34:08.600903 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:34:08.600909 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:34:08.601379 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:34:08.601736 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:08.601786 (+ 50us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:08.602870 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:08.602873 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:08.603903 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:08.603914 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:08.604515 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:08.604518 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:08.604642 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:08.605180 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:34:08.605195 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:34:08.606677 (+ 1482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:34:08.608406 (+ 1729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:08.608415 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:08.608424 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:08.608796 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:08.609022 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:08.609024 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:08.609026 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:08.609202 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:08.609592 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:34:08.609597 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:34:08.609865 (+ 268us) client_negotiation.cc:769] Sending connection context
0808 22:34:08.610053 (+ 188us) client_negotiation.cc:240] Negotiation successful
0808 22:34:08.610342 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":93,"threads_started":1}
I20260808 22:34:08.613130 20827 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.87.189" port: 43003 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 22:34:08 dist-test-slave-266d krb5kdc[20503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228448, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for kudu/127.31.87.188@KRBTEST.COM
Aug 08 22:34:08 dist-test-slave-266d krb5kdc[20503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228448, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.188@KRBTEST.COM for kudu/127.31.87.189@KRBTEST.COM
Aug 08 22:34:08 dist-test-slave-266d krb5kdc[20503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228447, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:34:08.620196 20632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:08.610949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.188:46191 (local address 127.0.0.1:47194)
0808 22:34:08.611131 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:08.611145 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:34:08.611218 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:34:08.611636 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:34:08.611640 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:34:08.611811 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:34:08.611905 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:08.611911 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:08.612730 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:08.612733 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:08.613470 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:08.613481 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:08.614133 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:08.614136 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:08.614211 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:08.614595 (+ 384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:34:08.614608 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:34:08.615923 (+ 1315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:34:08.618540 (+ 2617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:08.618542 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:08.618544 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:08.618783 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:08.619153 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:08.619157 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:08.619159 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:08.619253 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:08.619665 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:34:08.619669 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:34:08.619806 (+ 137us) client_negotiation.cc:769] Sending connection context
0808 22:34:08.619909 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 22:34:08.620038 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107}
I20260808 22:34:08.620304 20833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:08.611005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47194 (local address 127.31.87.188:46191)
0808 22:34:08.611431 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:08.611441 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:08.611454 (+ 13us) server_negotiation.cc:407] Connection header received
0808 22:34:08.611488 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:08.611491 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:08.611532 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:08.611608 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:34:08.612001 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:08.612637 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:08.613703 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:08.614019 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:08.614136 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:08.616038 (+ 1902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:34:08.616041 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:34:08.616044 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:34:08.616066 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:34:08.618425 (+ 2359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:08.618888 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:08.618892 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:08.618894 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:08.618979 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:08.619372 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:08.619376 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:08.619377 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:08.619582 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:34:08.619673 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:08.619970 (+ 297us) server_negotiation.cc:299] Negotiation successful
0808 22:34:08.620082 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":148,"threads_started":1}
I20260808 22:34:08.622225 20728 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.87.188" port: 46191 } attempt: 3
I20260808 22:34:08.622908 20731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:08.613792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51006 (local address 127.31.87.189:43003)
0808 22:34:08.613908 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:08.613913 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:08.613972 (+ 59us) server_negotiation.cc:407] Connection header received
0808 22:34:08.614085 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:08.614088 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:08.614146 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:08.614200 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:34:08.614740 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:08.615250 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:08.616305 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:08.616631 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:08.616711 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:08.618981 (+ 2270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:34:08.618984 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:34:08.618987 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:34:08.619011 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:34:08.620546 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:08.621032 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:08.621036 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:08.621038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:08.621148 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:08.622020 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:08.622024 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:08.622026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:08.622226 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:34:08.622297 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:08.622641 (+ 344us) server_negotiation.cc:299] Negotiation successful
0808 22:34:08.622754 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 22:34:08.623327 20829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:08.613543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.189:43003 (local address 127.0.0.1:51006)
0808 22:34:08.613906 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:08.613916 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 22:34:08.613984 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:34:08.614342 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:34:08.614345 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:34:08.614496 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:34:08.614594 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:08.614600 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:08.615365 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:08.615367 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:08.616155 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:08.616166 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:08.617019 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:08.617022 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:08.617133 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:08.617479 (+ 346us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:34:08.617494 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:34:08.618859 (+ 1365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:34:08.620699 (+ 1840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:08.620702 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:08.620704 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:08.620943 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:08.621836 (+ 893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:08.621839 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:08.621840 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:08.621914 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:08.622375 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:34:08.622378 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:34:08.622467 (+ 89us) client_negotiation.cc:769] Sending connection context
0808 22:34:08.623067 (+ 600us) client_negotiation.cc:240] Negotiation successful
0808 22:34:08.623191 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306}
I20260808 22:34:08.623903 20628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22619f565175445b8e75aa31f358ff31: Bootstrap starting.
I20260808 22:34:08.623980 20827 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.87.188" port: 46191 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:34:08.625001 20631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:08.613622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39086 (local address 127.31.87.190:45141)
0808 22:34:08.613715 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:08.613719 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:08.613731 (+ 12us) server_negotiation.cc:407] Connection header received
0808 22:34:08.613809 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:08.613812 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:08.613962 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:08.614037 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:34:08.614719 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:08.615395 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:08.616417 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:08.616639 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:08.616723 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:08.619543 (+ 2820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:34:08.619547 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:34:08.619550 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:34:08.619576 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:34:08.621717 (+ 2141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:08.622191 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:08.622194 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:08.622196 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:08.623196 (+ 1000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:08.623708 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:08.623712 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:08.623713 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:08.623914 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:34:08.624098 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:08.624756 (+ 658us) server_negotiation.cc:299] Negotiation successful
0808 22:34:08.624865 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 22:34:08.627254 20628 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 22619f565175445b8e75aa31f358ff31: Neither blocks nor log segments found. Creating new log.
Aug 08 22:34:08 dist-test-slave-266d krb5kdc[20503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228448, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.189@KRBTEST.COM for kudu/127.31.87.188@KRBTEST.COM
I20260808 22:34:08.628175 20628 log.cc:824] T 00000000000000000000000000000000 P 22619f565175445b8e75aa31f358ff31: Log is configured to *not* fsync() on all Append() calls
Aug 08 22:34:08 dist-test-slave-266d krb5kdc[20503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228448, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.188@KRBTEST.COM for kudu/127.31.87.188@KRBTEST.COM
I20260808 22:34:08.630373 20628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22619f565175445b8e75aa31f358ff31: No bootstrap required, opened a new log
I20260808 22:34:08.631589 20732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:08.622629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.188:46191 (local address 127.0.0.1:47200)
0808 22:34:08.622731 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:08.622744 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:34:08.622809 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:34:08.623269 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:34:08.623273 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:34:08.623427 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:34:08.623519 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:08.623526 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:08.624987 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:08.624990 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:08.625745 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:08.625757 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:08.626336 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:08.626340 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:08.626416 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:08.626830 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:34:08.626844 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:34:08.628213 (+ 1369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:34:08.630066 (+ 1853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:08.630069 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:08.630071 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:08.630311 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:08.630617 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:08.630620 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:08.630622 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:08.630726 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:08.631151 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:34:08.631154 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:34:08.631244 (+ 90us) client_negotiation.cc:769] Sending connection context
0808 22:34:08.631322 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 22:34:08.631437 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"spinlock_wait_cycles":2944}
I20260808 22:34:08.632159 20833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:08.622933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47200 (local address 127.31.87.188:46191)
0808 22:34:08.623046 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:08.623049 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:08.623062 (+ 13us) server_negotiation.cc:407] Connection header received
0808 22:34:08.623093 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:08.623096 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:08.623145 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:08.623767 (+ 622us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:34:08.623858 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:08.624861 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:08.625888 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:08.626207 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:08.626613 (+ 406us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:08.628356 (+ 1743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:34:08.628360 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:34:08.628362 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:34:08.628385 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:34:08.629944 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:08.630436 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:08.630439 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:08.630441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:08.630517 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:08.630840 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:08.630843 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:08.630844 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:08.631026 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:34:08.631834 (+ 808us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:08.631909 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 22:34:08.632015 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
Aug 08 22:34:08 dist-test-slave-266d krb5kdc[20503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228447, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.87.189@KRBTEST.COM
I20260808 22:34:08.633541 20832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:08.624586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47212 (local address 127.31.87.188:46191)
0808 22:34:08.624671 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:08.624675 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:08.624785 (+ 110us) server_negotiation.cc:407] Connection header received
0808 22:34:08.624912 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:08.624915 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:08.624947 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:08.624984 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:34:08.625615 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:08.626416 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:08.627412 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:08.627715 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:08.627799 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:08.629987 (+ 2188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:34:08.629990 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:34:08.629992 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:34:08.630013 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:34:08.631548 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:08.632386 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:08.632389 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:08.632391 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:08.632472 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:08.632752 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:08.632755 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:08.632757 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:08.632945 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:34:08.633039 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:08.633322 (+ 283us) server_negotiation.cc:299] Negotiation successful
0808 22:34:08.633421 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 22:34:08.633924 20829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:08.624414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.188:46191 (local address 127.0.0.1:47212)
0808 22:34:08.624725 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:08.624736 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:34:08.624830 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:34:08.625296 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:34:08.625299 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:34:08.625441 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:34:08.625508 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:08.625514 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:08.626516 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:08.626519 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:08.627239 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:08.627250 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:08.627967 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:08.627971 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:08.628043 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:08.628423 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:34:08.628437 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:34:08.629878 (+ 1441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:34:08.631873 (+ 1995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:08.631876 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:08.631878 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:08.632109 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:08.632574 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:08.632576 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:08.632578 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:08.632650 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:08.633037 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:34:08.633040 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:34:08.633158 (+ 118us) client_negotiation.cc:769] Sending connection context
0808 22:34:08.633692 (+ 534us) client_negotiation.cc:240] Negotiation successful
0808 22:34:08.633805 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272}
I20260808 22:34:08.634371 20628 raft_consensus.cc:348] T 00000000000000000000000000000000 P 22619f565175445b8e75aa31f358ff31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22619f565175445b8e75aa31f358ff31" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45141 } } peers { permanent_uuid: "c8dadf19214245e4806db1f1b1639dab" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 43003 } } peers { permanent_uuid: "57682fab1aa14cd8bf0bbbddc86da4dc" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 46191 } }
I20260808 22:34:08.634563 20628 raft_consensus.cc:374] T 00000000000000000000000000000000 P 22619f565175445b8e75aa31f358ff31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:34:08.634624 20628 raft_consensus.cc:729] T 00000000000000000000000000000000 P 22619f565175445b8e75aa31f358ff31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22619f565175445b8e75aa31f358ff31, State: Initialized, Role: FOLLOWER
I20260808 22:34:08.635128 20728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8dadf19214245e4806db1f1b1639dab: Bootstrap starting.
I20260808 22:34:08.635069 20628 consensus_queue.cc:260] T 00000000000000000000000000000000 P 22619f565175445b8e75aa31f358ff31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22619f565175445b8e75aa31f358ff31" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45141 } } peers { permanent_uuid: "c8dadf19214245e4806db1f1b1639dab" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 43003 } } peers { permanent_uuid: "57682fab1aa14cd8bf0bbbddc86da4dc" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 46191 } }
I20260808 22:34:08.635541 20843 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22619f565175445b8e75aa31f358ff31 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22619f565175445b8e75aa31f358ff31" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45141 } } peers { permanent_uuid: "c8dadf19214245e4806db1f1b1639dab" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 43003 } } peers { permanent_uuid: "57682fab1aa14cd8bf0bbbddc86da4dc" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 46191 } } }
I20260808 22:34:08.635659 20843 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22619f565175445b8e75aa31f358ff31 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:34:08.636394 20628 sys_catalog.cc:564] T 00000000000000000000000000000000 P 22619f565175445b8e75aa31f358ff31 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:34:08.636560 20827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57682fab1aa14cd8bf0bbbddc86da4dc: Bootstrap starting.
I20260808 22:34:08.637226 20728 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c8dadf19214245e4806db1f1b1639dab: Neither blocks nor log segments found. Creating new log.
I20260808 22:34:08.637977 20728 log.cc:824] T 00000000000000000000000000000000 P c8dadf19214245e4806db1f1b1639dab: Log is configured to *not* fsync() on all Append() calls
I20260808 22:34:08.640321 20827 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57682fab1aa14cd8bf0bbbddc86da4dc: Neither blocks nor log segments found. Creating new log.
I20260808 22:34:08.640446 20728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8dadf19214245e4806db1f1b1639dab: No bootstrap required, opened a new log
I20260808 22:34:08.641243 20827 log.cc:824] T 00000000000000000000000000000000 P 57682fab1aa14cd8bf0bbbddc86da4dc: Log is configured to *not* fsync() on all Append() calls
W20260808 22:34:08.641634 20855 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 22619f565175445b8e75aa31f358ff31: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:34:08.641705 20855 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260808 22:34:08.642040 20561 tablet.cc:2369] T 00000000000000000000000000000000 P 22619f565175445b8e75aa31f358ff31: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:34:08.643651 20827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57682fab1aa14cd8bf0bbbddc86da4dc: No bootstrap required, opened a new log
I20260808 22:34:08.644698 20731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:08.626246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51014 (local address 127.31.87.189:43003)
0808 22:34:08.626336 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:08.626339 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:08.626351 (+ 12us) server_negotiation.cc:407] Connection header received
0808 22:34:08.626467 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:08.626470 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:08.626511 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:08.626828 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:34:08.627586 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:08.628449 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:08.629963 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:08.630527 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:08.630601 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:08.638418 (+ 7817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:34:08.638422 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:34:08.638424 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:34:08.638442 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:34:08.642594 (+ 4152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:08.643060 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:08.643063 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:08.643065 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:08.643147 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:08.643468 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:08.643471 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:08.643473 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:08.643665 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:34:08.644381 (+ 716us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:08.644455 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 22:34:08.644554 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35,"spinlock_wait_cycles":4992}
I20260808 22:34:08.642168 20527 master_runner.cc:186] Error getting master registration for 127.31.87.189:43003: OK, instance_id { permanent_uuid: "c8dadf19214245e4806db1f1b1639dab" instance_seqno: 1786228448375377 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 22:34:08.645841 20527 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 22:34:08.647512 32094 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:34:08.647887 20827 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57682fab1aa14cd8bf0bbbddc86da4dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22619f565175445b8e75aa31f358ff31" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45141 } } peers { permanent_uuid: "c8dadf19214245e4806db1f1b1639dab" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 43003 } } peers { permanent_uuid: "57682fab1aa14cd8bf0bbbddc86da4dc" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 46191 } }
I20260808 22:34:08.648283 20827 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57682fab1aa14cd8bf0bbbddc86da4dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:34:08.648384 20827 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57682fab1aa14cd8bf0bbbddc86da4dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57682fab1aa14cd8bf0bbbddc86da4dc, State: Initialized, Role: FOLLOWER
I20260808 22:34:08.648497 20728 raft_consensus.cc:348] T 00000000000000000000000000000000 P c8dadf19214245e4806db1f1b1639dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22619f565175445b8e75aa31f358ff31" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45141 } } peers { permanent_uuid: "c8dadf19214245e4806db1f1b1639dab" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 43003 } } peers { permanent_uuid: "57682fab1aa14cd8bf0bbbddc86da4dc" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 46191 } }
I20260808 22:34:08.648696 20728 raft_consensus.cc:374] T 00000000000000000000000000000000 P c8dadf19214245e4806db1f1b1639dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:34:08.648777 20728 raft_consensus.cc:729] T 00000000000000000000000000000000 P c8dadf19214245e4806db1f1b1639dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8dadf19214245e4806db1f1b1639dab, State: Initialized, Role: FOLLOWER
I20260808 22:34:08.649034 20827 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57682fab1aa14cd8bf0bbbddc86da4dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22619f565175445b8e75aa31f358ff31" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45141 } } peers { permanent_uuid: "c8dadf19214245e4806db1f1b1639dab" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 43003 } } peers { permanent_uuid: "57682fab1aa14cd8bf0bbbddc86da4dc" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 46191 } }
I20260808 22:34:08.649215 20728 consensus_queue.cc:260] T 00000000000000000000000000000000 P c8dadf19214245e4806db1f1b1639dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22619f565175445b8e75aa31f358ff31" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45141 } } peers { permanent_uuid: "c8dadf19214245e4806db1f1b1639dab" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 43003 } } peers { permanent_uuid: "57682fab1aa14cd8bf0bbbddc86da4dc" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 46191 } }
I20260808 22:34:08.650420 20728 sys_catalog.cc:564] T 00000000000000000000000000000000 P c8dadf19214245e4806db1f1b1639dab [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:34:08.650646 20827 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57682fab1aa14cd8bf0bbbddc86da4dc [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:34:08.659317 20870 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8dadf19214245e4806db1f1b1639dab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22619f565175445b8e75aa31f358ff31" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45141 } } peers { permanent_uuid: "c8dadf19214245e4806db1f1b1639dab" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 43003 } } peers { permanent_uuid: "57682fab1aa14cd8bf0bbbddc86da4dc" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 46191 } } }
I20260808 22:34:08.659436 20870 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8dadf19214245e4806db1f1b1639dab [sys.catalog]: This master's current role is: FOLLOWER
W20260808 22:34:08.659369 20904 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 57682fab1aa14cd8bf0bbbddc86da4dc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:34:08.662266 20904 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:34:08.663589 20637 master_runner.cc:263] No leader master found from master c8dadf19214245e4806db1f1b1639dab
I20260808 22:34:08.663653 20637 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 22:34:08.663735 20735 master_runner.cc:263] No leader master found from master 57682fab1aa14cd8bf0bbbddc86da4dc
I20260808 22:34:08.663790 20735 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 22:34:08.664204 20868 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57682fab1aa14cd8bf0bbbddc86da4dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22619f565175445b8e75aa31f358ff31" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45141 } } peers { permanent_uuid: "c8dadf19214245e4806db1f1b1639dab" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 43003 } } peers { permanent_uuid: "57682fab1aa14cd8bf0bbbddc86da4dc" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 46191 } } }
I20260808 22:34:08.664372 20868 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57682fab1aa14cd8bf0bbbddc86da4dc [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:34:08.666162 20731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:08.649816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51020 (local address 127.31.87.189:43003)
0808 22:34:08.649908 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:08.649912 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:08.651410 (+ 1498us) server_negotiation.cc:407] Connection header received
0808 22:34:08.651588 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:08.651591 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:08.651630 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:08.651675 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:34:08.652652 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:08.653322 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:08.654396 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:08.654670 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:08.654738 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:08.661926 (+ 7188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:34:08.661930 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:34:08.661933 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:34:08.661953 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:34:08.663866 (+ 1913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:08.664546 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:08.664550 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:08.664552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:08.664638 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:08.665224 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:08.665227 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:08.665229 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:08.665435 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:34:08.665535 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:08.665902 (+ 367us) server_negotiation.cc:299] Negotiation successful
0808 22:34:08.666016 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 22:34:08.667578 20631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:08.650818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39100 (local address 127.31.87.190:45141)
0808 22:34:08.650915 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:08.650918 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:08.655483 (+ 4565us) server_negotiation.cc:407] Connection header received
0808 22:34:08.655638 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:08.655641 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:08.655682 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:08.655726 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:34:08.656618 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:08.657307 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:08.658296 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:08.658581 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:08.658652 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:08.664096 (+ 5444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:34:08.664100 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:34:08.664103 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:34:08.664129 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:34:08.665618 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:08.666048 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:08.666051 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:08.666053 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:08.666130 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:08.666626 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:08.666630 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:08.666631 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:08.666849 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:34:08.666953 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:08.667319 (+ 366us) server_negotiation.cc:299] Negotiation successful
0808 22:34:08.667429 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 22:34:08.669018 20832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:08.650379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47214 (local address 127.31.87.188:46191)
0808 22:34:08.650483 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:08.650486 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:08.653067 (+ 2581us) server_negotiation.cc:407] Connection header received
0808 22:34:08.653280 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:08.653283 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:08.653327 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:08.653376 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:34:08.654266 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:08.655008 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:08.659578 (+ 4570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:08.659911 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:08.659981 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:08.661978 (+ 1997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:34:08.661982 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:34:08.661985 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:34:08.662010 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:34:08.663530 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:08.667676 (+ 4146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:08.667679 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:08.667681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:08.667783 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:08.668079 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:08.668083 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:08.668084 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:08.668250 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:34:08.668345 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:08.668782 (+ 437us) server_negotiation.cc:299] Negotiation successful
0808 22:34:08.668890 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260808 22:34:08.673537 20898 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c8dadf19214245e4806db1f1b1639dab: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:34:08.673620 20898 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260808 22:34:08.834751 20765 tablet.cc:2369] T 00000000000000000000000000000000 P 57682fab1aa14cd8bf0bbbddc86da4dc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:34:08.857141 20868 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57682fab1aa14cd8bf0bbbddc86da4dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:34:08.857358 20868 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57682fab1aa14cd8bf0bbbddc86da4dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22619f565175445b8e75aa31f358ff31" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45141 } } peers { permanent_uuid: "c8dadf19214245e4806db1f1b1639dab" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 43003 } } peers { permanent_uuid: "57682fab1aa14cd8bf0bbbddc86da4dc" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 46191 } }
I20260808 22:34:08.858573 20868 leader_election.cc:288] T 00000000000000000000000000000000 P 57682fab1aa14cd8bf0bbbddc86da4dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22619f565175445b8e75aa31f358ff31 (127.31.87.190:45141), c8dadf19214245e4806db1f1b1639dab (127.31.87.189:43003)
I20260808 22:34:08.859011 20688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "57682fab1aa14cd8bf0bbbddc86da4dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8dadf19214245e4806db1f1b1639dab" is_pre_election: true
I20260808 22:34:08.859067 20591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "57682fab1aa14cd8bf0bbbddc86da4dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22619f565175445b8e75aa31f358ff31" is_pre_election: true
I20260808 22:34:08.859292 20688 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c8dadf19214245e4806db1f1b1639dab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57682fab1aa14cd8bf0bbbddc86da4dc in term 0.
I20260808 22:34:08.859436 20591 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 22619f565175445b8e75aa31f358ff31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57682fab1aa14cd8bf0bbbddc86da4dc in term 0.
I20260808 22:34:08.859714 20763 leader_election.cc:302] T 00000000000000000000000000000000 P 57682fab1aa14cd8bf0bbbddc86da4dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57682fab1aa14cd8bf0bbbddc86da4dc, c8dadf19214245e4806db1f1b1639dab; no voters:
I20260808 22:34:08.859990 20868 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57682fab1aa14cd8bf0bbbddc86da4dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:34:08.860080 20868 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57682fab1aa14cd8bf0bbbddc86da4dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:34:08.860123 20868 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57682fab1aa14cd8bf0bbbddc86da4dc [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:34:08.861284 20868 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57682fab1aa14cd8bf0bbbddc86da4dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22619f565175445b8e75aa31f358ff31" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45141 } } peers { permanent_uuid: "c8dadf19214245e4806db1f1b1639dab" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 43003 } } peers { permanent_uuid: "57682fab1aa14cd8bf0bbbddc86da4dc" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 46191 } }
I20260808 22:34:08.861614 20868 leader_election.cc:288] T 00000000000000000000000000000000 P 57682fab1aa14cd8bf0bbbddc86da4dc [CANDIDATE]: Term 1 election: Requested vote from peers 22619f565175445b8e75aa31f358ff31 (127.31.87.190:45141), c8dadf19214245e4806db1f1b1639dab (127.31.87.189:43003)
I20260808 22:34:08.861899 20688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "57682fab1aa14cd8bf0bbbddc86da4dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8dadf19214245e4806db1f1b1639dab"
I20260808 22:34:08.862023 20591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "57682fab1aa14cd8bf0bbbddc86da4dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22619f565175445b8e75aa31f358ff31"
I20260808 22:34:08.862051 20688 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c8dadf19214245e4806db1f1b1639dab [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:34:08.862146 20591 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 22619f565175445b8e75aa31f358ff31 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:34:08.863113 20688 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c8dadf19214245e4806db1f1b1639dab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57682fab1aa14cd8bf0bbbddc86da4dc in term 1.
I20260808 22:34:08.863224 20591 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 22619f565175445b8e75aa31f358ff31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57682fab1aa14cd8bf0bbbddc86da4dc in term 1.
I20260808 22:34:08.863410 20763 leader_election.cc:302] T 00000000000000000000000000000000 P 57682fab1aa14cd8bf0bbbddc86da4dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57682fab1aa14cd8bf0bbbddc86da4dc, c8dadf19214245e4806db1f1b1639dab; no voters:
I20260808 22:34:08.863577 20868 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57682fab1aa14cd8bf0bbbddc86da4dc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:34:08.863797 20868 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57682fab1aa14cd8bf0bbbddc86da4dc [term 1 LEADER]: Becoming Leader. State: Replica: 57682fab1aa14cd8bf0bbbddc86da4dc, State: Running, Role: LEADER
I20260808 22:34:08.864084 20868 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57682fab1aa14cd8bf0bbbddc86da4dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22619f565175445b8e75aa31f358ff31" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45141 } } peers { permanent_uuid: "c8dadf19214245e4806db1f1b1639dab" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 43003 } } peers { permanent_uuid: "57682fab1aa14cd8bf0bbbddc86da4dc" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 46191 } }
I20260808 22:34:08.866087 20908 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57682fab1aa14cd8bf0bbbddc86da4dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57682fab1aa14cd8bf0bbbddc86da4dc. Latest consensus state: current_term: 1 leader_uuid: "57682fab1aa14cd8bf0bbbddc86da4dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22619f565175445b8e75aa31f358ff31" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45141 } } peers { permanent_uuid: "c8dadf19214245e4806db1f1b1639dab" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 43003 } } peers { permanent_uuid: "57682fab1aa14cd8bf0bbbddc86da4dc" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 46191 } } }
I20260808 22:34:08.866201 20908 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57682fab1aa14cd8bf0bbbddc86da4dc [sys.catalog]: This master's current role is: LEADER
I20260808 22:34:08.866539 20910 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:34:08.867367 20910 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260808 22:34:08.871757 20662 tablet.cc:2369] T 00000000000000000000000000000000 P c8dadf19214245e4806db1f1b1639dab: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:34:08.873404 20688 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c8dadf19214245e4806db1f1b1639dab [term 1 FOLLOWER]: Refusing update from remote peer 57682fab1aa14cd8bf0bbbddc86da4dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:34:08.873433 20591 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 22619f565175445b8e75aa31f358ff31 [term 1 FOLLOWER]: Refusing update from remote peer 57682fab1aa14cd8bf0bbbddc86da4dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:34:08.874084 20908 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 57682fab1aa14cd8bf0bbbddc86da4dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8dadf19214245e4806db1f1b1639dab" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 43003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:34:08.874459 20908 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 57682fab1aa14cd8bf0bbbddc86da4dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "22619f565175445b8e75aa31f358ff31" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:34:08.877847 20870 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8dadf19214245e4806db1f1b1639dab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57682fab1aa14cd8bf0bbbddc86da4dc. Latest consensus state: current_term: 1 leader_uuid: "57682fab1aa14cd8bf0bbbddc86da4dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22619f565175445b8e75aa31f358ff31" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45141 } } peers { permanent_uuid: "c8dadf19214245e4806db1f1b1639dab" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 43003 } } peers { permanent_uuid: "57682fab1aa14cd8bf0bbbddc86da4dc" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 46191 } } }
I20260808 22:34:08.877974 20870 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8dadf19214245e4806db1f1b1639dab [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:34:08.877988 20843 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22619f565175445b8e75aa31f358ff31 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57682fab1aa14cd8bf0bbbddc86da4dc. Latest consensus state: current_term: 1 leader_uuid: "57682fab1aa14cd8bf0bbbddc86da4dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22619f565175445b8e75aa31f358ff31" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45141 } } peers { permanent_uuid: "c8dadf19214245e4806db1f1b1639dab" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 43003 } } peers { permanent_uuid: "57682fab1aa14cd8bf0bbbddc86da4dc" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 46191 } } }
I20260808 22:34:08.878106 20843 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22619f565175445b8e75aa31f358ff31 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:34:08.880944 20868 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57682fab1aa14cd8bf0bbbddc86da4dc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "57682fab1aa14cd8bf0bbbddc86da4dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22619f565175445b8e75aa31f358ff31" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45141 } } peers { permanent_uuid: "c8dadf19214245e4806db1f1b1639dab" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 43003 } } peers { permanent_uuid: "57682fab1aa14cd8bf0bbbddc86da4dc" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 46191 } } }
I20260808 22:34:08.880975 20908 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57682fab1aa14cd8bf0bbbddc86da4dc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "57682fab1aa14cd8bf0bbbddc86da4dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22619f565175445b8e75aa31f358ff31" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45141 } } peers { permanent_uuid: "c8dadf19214245e4806db1f1b1639dab" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 43003 } } peers { permanent_uuid: "57682fab1aa14cd8bf0bbbddc86da4dc" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 46191 } } }
I20260808 22:34:08.881049 20868 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57682fab1aa14cd8bf0bbbddc86da4dc [sys.catalog]: This master's current role is: LEADER
I20260808 22:34:08.881121 20908 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57682fab1aa14cd8bf0bbbddc86da4dc [sys.catalog]: This master's current role is: LEADER
I20260808 22:34:08.881855 20915 mvcc.cc:205] Tried to move back new op lower bound from 7316391726576152576 to 7316391726549766144. Current Snapshot: MvccSnapshot[applied={T|T < 7316391726576152576}]
I20260808 22:34:08.882179 20870 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8dadf19214245e4806db1f1b1639dab [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "57682fab1aa14cd8bf0bbbddc86da4dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22619f565175445b8e75aa31f358ff31" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45141 } } peers { permanent_uuid: "c8dadf19214245e4806db1f1b1639dab" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 43003 } } peers { permanent_uuid: "57682fab1aa14cd8bf0bbbddc86da4dc" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 46191 } } }
I20260808 22:34:08.882292 20870 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8dadf19214245e4806db1f1b1639dab [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:34:08.882368 20914 mvcc.cc:205] Tried to move back new op lower bound from 7316391726576152576 to 7316391726549766144. Current Snapshot: MvccSnapshot[applied={T|T < 7316391726576152576 or (T in {7316391726576152576})}]
I20260808 22:34:08.882493 20910 catalog_manager.cc:1347] Generated new cluster ID: 4948a8282828447082b70d3a081c9f8f
I20260808 22:34:08.882443 20843 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22619f565175445b8e75aa31f358ff31 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "57682fab1aa14cd8bf0bbbddc86da4dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22619f565175445b8e75aa31f358ff31" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 45141 } } peers { permanent_uuid: "c8dadf19214245e4806db1f1b1639dab" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 43003 } } peers { permanent_uuid: "57682fab1aa14cd8bf0bbbddc86da4dc" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 46191 } } }
I20260808 22:34:08.882551 20910 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:34:08.882544 20843 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22619f565175445b8e75aa31f358ff31 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:34:08.892048 20910 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:34:08.893067 20910 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:34:08.907258 20910 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57682fab1aa14cd8bf0bbbddc86da4dc: Generated new TSK 0
I20260808 22:34:08.908079 20910 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:34:09.642985 20855 catalog_manager.cc:1259] Loaded cluster ID: 4948a8282828447082b70d3a081c9f8f
I20260808 22:34:09.643069 20855 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 22619f565175445b8e75aa31f358ff31: loading cluster ID for follower catalog manager: success
I20260808 22:34:09.644744 20855 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 22619f565175445b8e75aa31f358ff31: acquiring CA information for follower catalog manager: success
I20260808 22:34:09.645910 20855 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 22619f565175445b8e75aa31f358ff31: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 22:34:09.674834 20898 catalog_manager.cc:1259] Loaded cluster ID: 4948a8282828447082b70d3a081c9f8f
I20260808 22:34:09.674911 20898 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c8dadf19214245e4806db1f1b1639dab: loading cluster ID for follower catalog manager: success
I20260808 22:34:09.676529 20898 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c8dadf19214245e4806db1f1b1639dab: acquiring CA information for follower catalog manager: success
I20260808 22:34:09.677682 20898 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c8dadf19214245e4806db1f1b1639dab: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 22:34:09.968842 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 20527
I20260808 22:34:09.980052 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 20637
I20260808 22:34:09.992947 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 20735
2026-08-08T22:34:09Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3409 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786228291013016-32094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:34:10 dist-test-slave-266d krb5kdc[20935](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:34:10 dist-test-slave-266d krb5kdc[20935](info): set up 2 sockets
Aug 08 22:34:10 dist-test-slave-266d krb5kdc[20935](info): commencing operation
krb5kdc: starting...
W20260808 22:34:11.131928 32094 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Aug 08 22:34:11 dist-test-slave-266d krb5kdc[20935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228451, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:34:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:34:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.87.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
I20260808 22:34:11.245935 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:34691
--webserver_interface=127.31.87.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:37471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:34691
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:34:11.354300 20952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:34:11.354523 20952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:34:11.354559 20952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:34:11.358659 20952 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:34:11.358719 20952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:34:11.358740 20952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:34:11.358759 20952 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:34:11.358779 20952 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:34:11.364020 20952 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:37471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786228291013016-32094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:34691
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:34691
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.87.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20952
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:34:11.364912 20952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:34:11.365861 20952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:34:11.373859 20961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:34:11.373874 20965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:34:11.373956 20952 server_base.cc:1034] running on GCE node
W20260808 22:34:11.373852 20963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:34:11.374677 20952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:34:11.375682 20952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:34:11.376844 20952 hybrid_clock.cc:648] HybridClock initialized: now 1786228451376830 us; error 25 us; skew 500 ppm
Aug 08 22:34:11 dist-test-slave-266d krb5kdc[20935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228451, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:34:11.379383 20952 init.cc:377] Logged in from keytab as kudu/127.31.87.190@KRBTEST.COM (short username kudu)
I20260808 22:34:11.380195 20952 webserver.cc:460] Webserver started at http://127.31.87.190:38125/ using document root <none> and password file <none>
I20260808 22:34:11.380697 20952 fs_manager.cc:359] Metadata directory not provided
I20260808 22:34:11.380755 20952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:34:11.380949 20952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:34:11.382870 20952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786228291013016-32094-0/minicluster-data/master-0/data/instance:
uuid: "8809757d65864e77bd9d5e8793762cb9"
format_stamp: "Formatted at 2026-08-08 22:34:11 on dist-test-slave-266d"
server_key: "bd2edab2231ef221c6ac166004bc61b8"
server_key_iv: "542412a9d33af07492d05584c5193ae7"
server_key_version: "encryptionkey@0"
I20260808 22:34:11.383407 20952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786228291013016-32094-0/minicluster-data/master-0/wal/instance:
uuid: "8809757d65864e77bd9d5e8793762cb9"
format_stamp: "Formatted at 2026-08-08 22:34:11 on dist-test-slave-266d"
server_key: "bd2edab2231ef221c6ac166004bc61b8"
server_key_iv: "542412a9d33af07492d05584c5193ae7"
server_key_version: "encryptionkey@0"
I20260808 22:34:11.386839 20952 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260808 22:34:11.388991 20975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:11.390154 20952 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 22:34:11.390264 20952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "8809757d65864e77bd9d5e8793762cb9"
format_stamp: "Formatted at 2026-08-08 22:34:11 on dist-test-slave-266d"
server_key: "bd2edab2231ef221c6ac166004bc61b8"
server_key_iv: "542412a9d33af07492d05584c5193ae7"
server_key_version: "encryptionkey@0"
I20260808 22:34:11.390353 20952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:34:11.421960 20952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:34:11.424768 20952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:34:11.424960 20952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:34:11.432000 20952 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.190:34691
I20260808 22:34:11.432008 21049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.190:34691 every 8 connection(s)
I20260808 22:34:11.432940 20952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:34:11.435565 21051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:34:11.440701 21051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8809757d65864e77bd9d5e8793762cb9: Bootstrap starting.
I20260808 22:34:11.441402 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 20952
I20260808 22:34:11.441517 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786228291013016-32094-0/minicluster-data/master-0/wal/instance
I20260808 22:34:11.441766 32094 external_mini_cluster.cc:1428] Setting key 9704f0980934d80bec863c4a2e964b92
I20260808 22:34:11.443200 21051 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8809757d65864e77bd9d5e8793762cb9: Neither blocks nor log segments found. Creating new log.
I20260808 22:34:11.443863 21051 log.cc:824] T 00000000000000000000000000000000 P 8809757d65864e77bd9d5e8793762cb9: Log is configured to *not* fsync() on all Append() calls
I20260808 22:34:11.445586 21051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8809757d65864e77bd9d5e8793762cb9: No bootstrap required, opened a new log
I20260808 22:34:11.448233 21051 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8809757d65864e77bd9d5e8793762cb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8809757d65864e77bd9d5e8793762cb9" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 34691 } }
I20260808 22:34:11.448436 21051 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8809757d65864e77bd9d5e8793762cb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:34:11.448490 21051 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8809757d65864e77bd9d5e8793762cb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8809757d65864e77bd9d5e8793762cb9, State: Initialized, Role: FOLLOWER
I20260808 22:34:11.448889 21051 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8809757d65864e77bd9d5e8793762cb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8809757d65864e77bd9d5e8793762cb9" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 34691 } }
I20260808 22:34:11.449013 21051 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8809757d65864e77bd9d5e8793762cb9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:34:11.449075 21051 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8809757d65864e77bd9d5e8793762cb9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:34:11.449187 21051 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8809757d65864e77bd9d5e8793762cb9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:34:11.449918 21051 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8809757d65864e77bd9d5e8793762cb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8809757d65864e77bd9d5e8793762cb9" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 34691 } }
Aug 08 22:34:11 dist-test-slave-266d krb5kdc[20935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228451, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:34:11.450229 21051 leader_election.cc:302] T 00000000000000000000000000000000 P 8809757d65864e77bd9d5e8793762cb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8809757d65864e77bd9d5e8793762cb9; no voters:
I20260808 22:34:11.450489 21051 leader_election.cc:288] T 00000000000000000000000000000000 P 8809757d65864e77bd9d5e8793762cb9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:34:11.450603 21057 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8809757d65864e77bd9d5e8793762cb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:34:11.450809 21057 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8809757d65864e77bd9d5e8793762cb9 [term 1 LEADER]: Becoming Leader. State: Replica: 8809757d65864e77bd9d5e8793762cb9, State: Running, Role: LEADER
I20260808 22:34:11.451179 21057 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8809757d65864e77bd9d5e8793762cb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8809757d65864e77bd9d5e8793762cb9" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 34691 } }
I20260808 22:34:11.451597 21051 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8809757d65864e77bd9d5e8793762cb9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:34:11.452374 21057 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8809757d65864e77bd9d5e8793762cb9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8809757d65864e77bd9d5e8793762cb9. Latest consensus state: current_term: 1 leader_uuid: "8809757d65864e77bd9d5e8793762cb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8809757d65864e77bd9d5e8793762cb9" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 34691 } } }
I20260808 22:34:11.452489 21057 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8809757d65864e77bd9d5e8793762cb9 [sys.catalog]: This master's current role is: LEADER
I20260808 22:34:11.452773 21058 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8809757d65864e77bd9d5e8793762cb9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8809757d65864e77bd9d5e8793762cb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8809757d65864e77bd9d5e8793762cb9" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 34691 } } }
I20260808 22:34:11.452925 21058 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8809757d65864e77bd9d5e8793762cb9 [sys.catalog]: This master's current role is: LEADER
I20260808 22:34:11.453037 21067 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:34:11.456022 21067 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:34:11.456362 21054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:11.442822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41794 (local address 127.31.87.190:34691)
0808 22:34:11.443230 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:11.443244 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:11.443279 (+ 35us) server_negotiation.cc:407] Connection header received
0808 22:34:11.443905 (+ 626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:11.443921 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:11.444259 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:11.444545 (+ 286us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:34:11.445352 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:11.446337 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:11.447176 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:11.448032 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:11.448186 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:11.450688 (+ 2502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:34:11.450694 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:34:11.450710 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:34:11.450741 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:34:11.452825 (+ 2084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:11.453391 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:11.453397 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:11.453403 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:11.453551 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:11.453887 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:11.453890 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:11.453892 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:11.454389 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:34:11.454589 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:11.455194 (+ 605us) server_negotiation.cc:299] Negotiation successful
0808 22:34:11.455475 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":105,"threads_started":1}
I20260808 22:34:11.459442 32094 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:34:11.461776 21067 catalog_manager.cc:1347] Generated new cluster ID: 9550094d26d7445bb3a8215699ce0cd7
I20260808 22:34:11.461840 21067 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:34:11.484153 21067 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:34:11.484982 21067 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:34:11.495671 21067 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8809757d65864e77bd9d5e8793762cb9: Generated new TSK 0
I20260808 22:34:11.496263 21067 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:34:11.514976 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 20952
2026-08-08T22:34:11Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1524 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:34:11 dist-test-slave-266d krb5kdc[21091](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:34:11 dist-test-slave-266d krb5kdc[21091](info): set up 2 sockets
Aug 08 22:34:11 dist-test-slave-266d krb5kdc[21091](info): commencing operation
krb5kdc: starting...
W20260808 22:34:12.641742 32094 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Aug 08 22:34:12 dist-test-slave-266d krb5kdc[21091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:34:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:34:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.87.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
I20260808 22:34:12.756784 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:41657
--webserver_interface=127.31.87.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:43005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:41657
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:34:12.865743 21109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:34:12.865999 21109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:34:12.866063 21109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:34:12.869967 21109 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:34:12.870028 21109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:34:12.870060 21109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:34:12.870087 21109 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:34:12.870117 21109 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:34:12.875205 21109 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:43005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:41657
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:41657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.87.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21109
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:34:12.876178 21109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:34:12.877219 21109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:34:12.885288 21122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:34:12.885324 21118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:34:12.885361 21109 server_base.cc:1034] running on GCE node
W20260808 22:34:12.885290 21117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:34:12.886070 21109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:34:12.887126 21109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:34:12.888327 21109 hybrid_clock.cc:648] HybridClock initialized: now 1786228452888307 us; error 31 us; skew 500 ppm
Aug 08 22:34:12 dist-test-slave-266d krb5kdc[21091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228452, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:34:12.890901 21109 init.cc:377] Logged in from keytab as kudu/127.31.87.190@KRBTEST.COM (short username kudu)
I20260808 22:34:12.891732 21109 webserver.cc:460] Webserver started at http://127.31.87.190:37393/ using document root <none> and password file <none>
I20260808 22:34:12.892236 21109 fs_manager.cc:359] Metadata directory not provided
I20260808 22:34:12.892308 21109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:34:12.892513 21109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:34:12.894471 21109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/master-0/data/instance:
uuid: "aa7651551d8d498890f0984c21549439"
format_stamp: "Formatted at 2026-08-08 22:34:12 on dist-test-slave-266d"
server_key: "8c08cb5db44f911c46876007b3896cb8"
server_key_iv: "4ab580b9b3a9561b384123b0f8a3a859"
server_key_version: "encryptionkey@0"
I20260808 22:34:12.894977 21109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/master-0/wal/instance:
uuid: "aa7651551d8d498890f0984c21549439"
format_stamp: "Formatted at 2026-08-08 22:34:12 on dist-test-slave-266d"
server_key: "8c08cb5db44f911c46876007b3896cb8"
server_key_iv: "4ab580b9b3a9561b384123b0f8a3a859"
server_key_version: "encryptionkey@0"
I20260808 22:34:12.898473 21109 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:34:12.900596 21131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:12.901738 21109 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:34:12.901847 21109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "aa7651551d8d498890f0984c21549439"
format_stamp: "Formatted at 2026-08-08 22:34:12 on dist-test-slave-266d"
server_key: "8c08cb5db44f911c46876007b3896cb8"
server_key_iv: "4ab580b9b3a9561b384123b0f8a3a859"
server_key_version: "encryptionkey@0"
I20260808 22:34:12.901932 21109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:34:12.927222 21109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:34:12.929718 21109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:34:12.929878 21109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:34:12.936401 21109 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.190:41657
I20260808 22:34:12.936654 21212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.190:41657 every 8 connection(s)
I20260808 22:34:12.937304 21109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:34:12.939854 21213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:34:12.942023 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 21109
I20260808 22:34:12.942162 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/master-0/wal/instance
I20260808 22:34:12.942418 32094 external_mini_cluster.cc:1428] Setting key a622e1779e65bb366cad4a2d99a34692
I20260808 22:34:12.945310 21213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439: Bootstrap starting.
I20260808 22:34:12.947712 21213 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439: Neither blocks nor log segments found. Creating new log.
I20260808 22:34:12.948565 21213 log.cc:824] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439: Log is configured to *not* fsync() on all Append() calls
I20260808 22:34:12.950395 21213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439: No bootstrap required, opened a new log
Aug 08 22:34:12 dist-test-slave-266d krb5kdc[21091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:34:12.953362 21213 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa7651551d8d498890f0984c21549439" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41657 } }
I20260808 22:34:12.953564 21213 raft_consensus.cc:374] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:34:12.953622 21213 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa7651551d8d498890f0984c21549439, State: Initialized, Role: FOLLOWER
I20260808 22:34:12.954016 21213 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa7651551d8d498890f0984c21549439" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41657 } }
I20260808 22:34:12.954152 21213 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:34:12.954214 21213 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:34:12.954308 21213 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:34:12.955082 21213 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa7651551d8d498890f0984c21549439" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41657 } }
I20260808 22:34:12.955394 21213 leader_election.cc:302] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa7651551d8d498890f0984c21549439; no voters:
I20260808 22:34:12.955657 21213 leader_election.cc:288] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:34:12.955776 21225 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:34:12.955978 21225 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439 [term 1 LEADER]: Becoming Leader. State: Replica: aa7651551d8d498890f0984c21549439, State: Running, Role: LEADER
I20260808 22:34:12.956358 21221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:12.943618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38464 (local address 127.31.87.190:41657)
0808 22:34:12.943997 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:12.944015 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:12.944208 (+ 193us) server_negotiation.cc:407] Connection header received
0808 22:34:12.944910 (+ 702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:12.944925 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:12.945276 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:12.945550 (+ 274us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:34:12.946395 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:12.947417 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:12.948553 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:12.949018 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:12.949182 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:12.951399 (+ 2217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:34:12.951405 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:34:12.951416 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:34:12.951442 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:34:12.953504 (+ 2062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:12.953922 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:12.953927 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:12.953931 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:12.954034 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:12.954289 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:12.954292 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:12.954294 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:12.954682 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:34:12.954862 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:12.955130 (+ 268us) server_negotiation.cc:299] Negotiation successful
0808 22:34:12.955423 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":106,"threads_started":1}
I20260808 22:34:12.956328 21225 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa7651551d8d498890f0984c21549439" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41657 } }
I20260808 22:34:12.956808 21213 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:34:12.958104 21228 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa7651551d8d498890f0984c21549439. Latest consensus state: current_term: 1 leader_uuid: "aa7651551d8d498890f0984c21549439" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa7651551d8d498890f0984c21549439" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41657 } } }
I20260808 22:34:12.958276 21228 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439 [sys.catalog]: This master's current role is: LEADER
I20260808 22:34:12.958627 21226 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aa7651551d8d498890f0984c21549439" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa7651551d8d498890f0984c21549439" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41657 } } }
I20260808 22:34:12.959348 21226 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439 [sys.catalog]: This master's current role is: LEADER
W20260808 22:34:12.961499 21248 catalog_manager.cc:1558] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:34:12.961638 21248 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:34:12.961863 21245 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:34:12.962549 21245 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:34:12.967636 21245 catalog_manager.cc:1347] Generated new cluster ID: 096d94a0830e4ccc8476217ffb722cc7
I20260808 22:34:12.967698 21245 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:34:12.990137 21245 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:34:12.990985 21245 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:34:12.999639 21245 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439: Generated new TSK 0
I20260808 22:34:13.000304 21245 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.87.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab.
Entry for principal kudu/127.31.87.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab.
I20260808 22:34:13.031608 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.129:0
--local_ip_for_outbound_sockets=127.31.87.129
--webserver_interface=127.31.87.129
--webserver_port=0
--tserver_master_addrs=127.31.87.190:41657
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab
--principal=kudu/127.31.87.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:43005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:34:13.141189 21258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:34:13.141428 21258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:34:13.141477 21258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:34:13.145457 21258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:34:13.145511 21258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:34:13.145606 21258 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.129
I20260808 22:34:13.150636 21258 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:43005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab
--principal=kudu/127.31.87.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.87.129
--webserver_port=0
--tserver_master_addrs=127.31.87.190:41657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21258
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.129
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:34:13.151551 21258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:34:13.152475 21258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:34:13.161826 21265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:34:13.161960 21258 server_base.cc:1034] running on GCE node
W20260808 22:34:13.161824 21266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:34:13.162127 21268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:34:13.162506 21258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:34:13.163111 21258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:34:13.164269 21258 hybrid_clock.cc:648] HybridClock initialized: now 1786228453164243 us; error 42 us; skew 500 ppm
Aug 08 22:34:13 dist-test-slave-266d krb5kdc[21091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228453, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:34:13.166908 21258 init.cc:377] Logged in from keytab as kudu/127.31.87.129@KRBTEST.COM (short username kudu)
I20260808 22:34:13.167642 21258 webserver.cc:460] Webserver started at http://127.31.87.129:34235/ using document root <none> and password file <none>
I20260808 22:34:13.168139 21258 fs_manager.cc:359] Metadata directory not provided
I20260808 22:34:13.168195 21258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:34:13.168355 21258 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:34:13.170215 21258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-0/data/instance:
uuid: "4355aa00a45b496793ff2602e299e94e"
format_stamp: "Formatted at 2026-08-08 22:34:13 on dist-test-slave-266d"
server_key: "fcb44dde4838b861fb64d9a0e7ef8919"
server_key_iv: "8a0b22cc07e34d0e72cb43451eaa5325"
server_key_version: "encryptionkey@0"
I20260808 22:34:13.170687 21258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-0/wal/instance:
uuid: "4355aa00a45b496793ff2602e299e94e"
format_stamp: "Formatted at 2026-08-08 22:34:13 on dist-test-slave-266d"
server_key: "fcb44dde4838b861fb64d9a0e7ef8919"
server_key_iv: "8a0b22cc07e34d0e72cb43451eaa5325"
server_key_version: "encryptionkey@0"
I20260808 22:34:13.174031 21258 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:34:13.176168 21285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:13.177376 21258 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:34:13.177480 21258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-0/wal
uuid: "4355aa00a45b496793ff2602e299e94e"
format_stamp: "Formatted at 2026-08-08 22:34:13 on dist-test-slave-266d"
server_key: "fcb44dde4838b861fb64d9a0e7ef8919"
server_key_iv: "8a0b22cc07e34d0e72cb43451eaa5325"
server_key_version: "encryptionkey@0"
I20260808 22:34:13.177575 21258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:34:13.205799 21258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:34:13.208542 21258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:34:13.208730 21258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:34:13.209292 21258 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:34:13.210068 21258 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:34:13.210117 21258 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:13.210168 21258 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:34:13.210197 21258 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:13.218159 21258 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.129:38117
I20260808 22:34:13.218178 21468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.129:38117 every 8 connection(s)
I20260808 22:34:13.218950 21258 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
I20260808 22:34:13.227483 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 21258
Aug 08 22:34:13 dist-test-slave-266d krb5kdc[21091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228453, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.129@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:34:13.227619 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-0/wal/instance
I20260808 22:34:13.227931 32094 external_mini_cluster.cc:1428] Setting key d69e67f46212924bd14ef38acdc5a333
I20260808 22:34:13.231530 21221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:13.220561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:48643 (local address 127.31.87.190:41657)
0808 22:34:13.220679 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:13.220683 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:13.221466 (+ 783us) server_negotiation.cc:407] Connection header received
0808 22:34:13.222742 (+ 1276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:13.222745 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:13.222803 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:13.222875 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:34:13.223961 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:13.224490 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:13.225615 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:13.225844 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:13.225934 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:13.228230 (+ 2296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:34:13.228233 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:34:13.228235 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:34:13.228260 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:34:13.229593 (+ 1333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:13.230109 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:13.230112 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:13.230113 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:13.230179 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:13.230488 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:13.230490 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:13.230491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:13.230642 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:34:13.230709 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:13.231226 (+ 517us) server_negotiation.cc:299] Negotiation successful
0808 22:34:13.231367 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 22:34:13.232298 21473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:13.220903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.129:48643)
0808 22:34:13.221332 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:13.221373 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 22:34:13.222540 (+ 1167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:34:13.223049 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:34:13.223055 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:34:13.223382 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:34:13.223822 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:13.223836 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:13.224604 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:13.224607 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:13.225500 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:13.225510 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:13.225980 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:13.225984 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:13.226137 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:13.226590 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:34:13.226605 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:34:13.228068 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:34:13.229706 (+ 1638us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:13.229712 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:13.229726 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:13.230022 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:13.230247 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:13.230250 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:13.230252 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:13.230414 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:13.230725 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:34:13.230731 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:34:13.231006 (+ 275us) client_negotiation.cc:769] Sending connection context
0808 22:34:13.231198 (+ 192us) client_negotiation.cc:240] Negotiation successful
0808 22:34:13.231506 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":119,"threads_started":1}
I20260808 22:34:13.233508 21469 heartbeater.cc:347] Connected to a master server at 127.31.87.190:41657
I20260808 22:34:13.233763 21469 heartbeater.cc:464] Registering TS with master...
I20260808 22:34:13.234323 21469 heartbeater.cc:511] Master 127.31.87.190:41657 requested a full tablet report, sending...
I20260808 22:34:13.235805 21159 ts_manager.cc:194] Registered new tserver with Master: 4355aa00a45b496793ff2602e299e94e (127.31.87.129:38117)
I20260808 22:34:13.236899 21159 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.129@KRBTEST.COM'} at 127.31.87.129:48643
WARNING: no policy specified for kudu/127.31.87.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab.
Entry for principal kudu/127.31.87.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab.
I20260808 22:34:13.253674 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.130:0
--local_ip_for_outbound_sockets=127.31.87.130
--webserver_interface=127.31.87.130
--webserver_port=0
--tserver_master_addrs=127.31.87.190:41657
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab
--principal=kudu/127.31.87.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:43005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:34:13.359972 21478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:34:13.360221 21478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:34:13.360283 21478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:34:13.364189 21478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:34:13.364241 21478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:34:13.364315 21478 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.130
I20260808 22:34:13.369290 21478 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:43005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab
--principal=kudu/127.31.87.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.87.130
--webserver_port=0
--tserver_master_addrs=127.31.87.190:41657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21478
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.130
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:34:13.370182 21478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:34:13.371084 21478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:34:13.379989 21486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:34:13.380059 21491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:34:13.380013 21478 server_base.cc:1034] running on GCE node
W20260808 22:34:13.380182 21487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:34:13.380540 21478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:34:13.381188 21478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:34:13.382351 21478 hybrid_clock.cc:648] HybridClock initialized: now 1786228453382331 us; error 30 us; skew 500 ppm
Aug 08 22:34:13 dist-test-slave-266d krb5kdc[21091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228453, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:34:13.384847 21478 init.cc:377] Logged in from keytab as kudu/127.31.87.130@KRBTEST.COM (short username kudu)
I20260808 22:34:13.385676 21478 webserver.cc:460] Webserver started at http://127.31.87.130:44951/ using document root <none> and password file <none>
I20260808 22:34:13.386183 21478 fs_manager.cc:359] Metadata directory not provided
I20260808 22:34:13.386241 21478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:34:13.386432 21478 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:34:13.388346 21478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-1/data/instance:
uuid: "0db1fc6a077a4abb8d1650a2b9540d75"
format_stamp: "Formatted at 2026-08-08 22:34:13 on dist-test-slave-266d"
server_key: "e06bdce1b696c3ead6da19debab55c73"
server_key_iv: "5cf8f2a82dbecb53ae41691fdad2c48f"
server_key_version: "encryptionkey@0"
I20260808 22:34:13.388861 21478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-1/wal/instance:
uuid: "0db1fc6a077a4abb8d1650a2b9540d75"
format_stamp: "Formatted at 2026-08-08 22:34:13 on dist-test-slave-266d"
server_key: "e06bdce1b696c3ead6da19debab55c73"
server_key_iv: "5cf8f2a82dbecb53ae41691fdad2c48f"
server_key_version: "encryptionkey@0"
I20260808 22:34:13.392263 21478 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 22:34:13.394488 21503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:13.395586 21478 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:34:13.395677 21478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-1/wal
uuid: "0db1fc6a077a4abb8d1650a2b9540d75"
format_stamp: "Formatted at 2026-08-08 22:34:13 on dist-test-slave-266d"
server_key: "e06bdce1b696c3ead6da19debab55c73"
server_key_iv: "5cf8f2a82dbecb53ae41691fdad2c48f"
server_key_version: "encryptionkey@0"
I20260808 22:34:13.395777 21478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:34:13.410693 21478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:34:13.413008 21478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:34:13.413182 21478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:34:13.413672 21478 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:34:13.414391 21478 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:34:13.414435 21478 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:13.414474 21478 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:34:13.414502 21478 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:13.422573 21478 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.130:41469
I20260808 22:34:13.422597 21667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.130:41469 every 8 connection(s)
I20260808 22:34:13.423347 21478 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
I20260808 22:34:13.428728 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 21478
I20260808 22:34:13.428851 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-1/wal/instance
I20260808 22:34:13.429136 32094 external_mini_cluster.cc:1428] Setting key ca41f6cb9cbce9c0fcf033f4909f7659
Aug 08 22:34:13 dist-test-slave-266d krb5kdc[21091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228453, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.130@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:34:13.436146 21221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:13.424892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:52283 (local address 127.31.87.190:41657)
0808 22:34:13.425000 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:13.425004 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:13.425759 (+ 755us) server_negotiation.cc:407] Connection header received
0808 22:34:13.427024 (+ 1265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:13.427028 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:13.427090 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:13.427161 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:34:13.428213 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:13.428874 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:13.430198 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:13.430467 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:13.430549 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:13.432856 (+ 2307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:34:13.432859 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:34:13.432861 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:34:13.432884 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:34:13.434268 (+ 1384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:13.434726 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:13.434729 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:13.434732 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:13.434828 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:13.435108 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:13.435112 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:13.435114 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:13.435339 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:34:13.435441 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:13.435862 (+ 421us) server_negotiation.cc:299] Negotiation successful
0808 22:34:13.435983 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 22:34:13.436774 21670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:13.425209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.130:52283)
0808 22:34:13.425628 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:13.425665 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 22:34:13.426827 (+ 1162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:34:13.427339 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:34:13.427345 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:34:13.427652 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:34:13.428072 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:13.428084 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:13.429007 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:13.429010 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:13.430062 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:13.430073 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:13.430605 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:13.430609 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:13.430751 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:13.431241 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:34:13.431257 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:34:13.432702 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:34:13.434384 (+ 1682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:13.434389 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:13.434400 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:13.434635 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:13.434903 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:13.434905 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:13.434906 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:13.435019 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:13.435441 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:34:13.435447 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:34:13.435648 (+ 201us) client_negotiation.cc:769] Sending connection context
0808 22:34:13.435787 (+ 139us) client_negotiation.cc:240] Negotiation successful
0808 22:34:13.436002 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":109,"threads_started":1}
I20260808 22:34:13.437769 21668 heartbeater.cc:347] Connected to a master server at 127.31.87.190:41657
I20260808 22:34:13.437968 21668 heartbeater.cc:464] Registering TS with master...
I20260808 22:34:13.438376 21668 heartbeater.cc:511] Master 127.31.87.190:41657 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.31.87.131@KRBTEST.COM; defaulting to no policy
I20260808 22:34:13.439338 21159 ts_manager.cc:194] Registered new tserver with Master: 0db1fc6a077a4abb8d1650a2b9540d75 (127.31.87.130:41469)
I20260808 22:34:13.439872 21159 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.130@KRBTEST.COM'} at 127.31.87.130:52283
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab.
Entry for principal kudu/127.31.87.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab.
I20260808 22:34:13.455041 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.131:0
--local_ip_for_outbound_sockets=127.31.87.131
--webserver_interface=127.31.87.131
--webserver_port=0
--tserver_master_addrs=127.31.87.190:41657
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab
--principal=kudu/127.31.87.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:43005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:34:13.562641 21673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:34:13.562865 21673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:34:13.562901 21673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:34:13.566728 21673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:34:13.566777 21673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:34:13.566831 21673 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.131
I20260808 22:34:13.571717 21673 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:43005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab
--principal=kudu/127.31.87.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.87.131
--webserver_port=0
--tserver_master_addrs=127.31.87.190:41657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21673
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.131
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:34:13.572538 21673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:34:13.573436 21673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:34:13.582384 21689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:34:13.582387 21686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:34:13.582485 21673 server_base.cc:1034] running on GCE node
W20260808 22:34:13.582386 21685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:34:13.583045 21673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:34:13.583626 21673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:34:13.584775 21673 hybrid_clock.cc:648] HybridClock initialized: now 1786228453584758 us; error 27 us; skew 500 ppm
Aug 08 22:34:13 dist-test-slave-266d krb5kdc[21091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228453, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:34:13.587275 21673 init.cc:377] Logged in from keytab as kudu/127.31.87.131@KRBTEST.COM (short username kudu)
I20260808 22:34:13.588029 21673 webserver.cc:460] Webserver started at http://127.31.87.131:42581/ using document root <none> and password file <none>
I20260808 22:34:13.588531 21673 fs_manager.cc:359] Metadata directory not provided
I20260808 22:34:13.588589 21673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:34:13.588773 21673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:34:13.590655 21673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-2/data/instance:
uuid: "d686840dc3584fbd9b3c7accdb6e26a3"
format_stamp: "Formatted at 2026-08-08 22:34:13 on dist-test-slave-266d"
server_key: "a7c67bcb19edb7a91b0d5b9b7bc6a0af"
server_key_iv: "481226f7429a5f096fd39bc0f0c6853b"
server_key_version: "encryptionkey@0"
I20260808 22:34:13.591152 21673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-2/wal/instance:
uuid: "d686840dc3584fbd9b3c7accdb6e26a3"
format_stamp: "Formatted at 2026-08-08 22:34:13 on dist-test-slave-266d"
server_key: "a7c67bcb19edb7a91b0d5b9b7bc6a0af"
server_key_iv: "481226f7429a5f096fd39bc0f0c6853b"
server_key_version: "encryptionkey@0"
I20260808 22:34:13.594513 21673 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260808 22:34:13.596669 21698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:13.597813 21673 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:34:13.597915 21673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-2/wal
uuid: "d686840dc3584fbd9b3c7accdb6e26a3"
format_stamp: "Formatted at 2026-08-08 22:34:13 on dist-test-slave-266d"
server_key: "a7c67bcb19edb7a91b0d5b9b7bc6a0af"
server_key_iv: "481226f7429a5f096fd39bc0f0c6853b"
server_key_version: "encryptionkey@0"
I20260808 22:34:13.598007 21673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:34:13.634424 21673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:34:13.636997 21673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:34:13.637195 21673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:34:13.637737 21673 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:34:13.638531 21673 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:34:13.638577 21673 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:13.638626 21673 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:34:13.638653 21673 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:13.646908 21673 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.131:44849
I20260808 22:34:13.646934 21887 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.131:44849 every 8 connection(s)
I20260808 22:34:13.647707 21673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
I20260808 22:34:13.651121 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 21673
I20260808 22:34:13.651247 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-2/wal/instance
I20260808 22:34:13.651518 32094 external_mini_cluster.cc:1428] Setting key 8dec51e133c79d83312771b151ec8a85
Aug 08 22:34:13 dist-test-slave-266d krb5kdc[21091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228453, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.131@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:34:13.659968 21221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:13.649355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:51329 (local address 127.31.87.190:41657)
0808 22:34:13.649487 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:13.649491 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:13.650248 (+ 757us) server_negotiation.cc:407] Connection header received
0808 22:34:13.651716 (+ 1468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:13.651719 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:13.651766 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:13.651837 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:34:13.653050 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:13.653654 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:13.654701 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:13.654906 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:13.654969 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:13.657198 (+ 2229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:34:13.657201 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:34:13.657203 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:34:13.657240 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:34:13.658434 (+ 1194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:13.658850 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:13.658853 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:13.658854 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:13.658922 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:13.659168 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:13.659170 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:13.659171 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:13.659321 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:34:13.659389 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:13.659762 (+ 373us) server_negotiation.cc:299] Negotiation successful
0808 22:34:13.659852 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 22:34:13.660676 21891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:13.649675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.131:51329)
0808 22:34:13.650101 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:13.650150 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 22:34:13.651496 (+ 1346us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:34:13.651993 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:34:13.652000 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:34:13.652397 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:34:13.652927 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:13.652946 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:13.653760 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:13.653763 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:13.654602 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:13.654614 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:13.655021 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:13.655024 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:13.655164 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:13.655650 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:34:13.655665 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:34:13.657039 (+ 1374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:34:13.658526 (+ 1487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:13.658531 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:13.658541 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:13.658776 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:13.658983 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:13.658985 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:13.658987 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:13.659098 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:13.659389 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:34:13.659393 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:34:13.659585 (+ 192us) client_negotiation.cc:769] Sending connection context
0808 22:34:13.659727 (+ 142us) client_negotiation.cc:240] Negotiation successful
0808 22:34:13.659938 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":113,"threads_started":1}
I20260808 22:34:13.661634 21889 heartbeater.cc:347] Connected to a master server at 127.31.87.190:41657
I20260808 22:34:13.661834 21889 heartbeater.cc:464] Registering TS with master...
I20260808 22:34:13.662249 21889 heartbeater.cc:511] Master 127.31.87.190:41657 requested a full tablet report, sending...
I20260808 22:34:13.663167 21159 ts_manager.cc:194] Registered new tserver with Master: d686840dc3584fbd9b3c7accdb6e26a3 (127.31.87.131:44849)
I20260808 22:34:13.663688 21159 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.131@KRBTEST.COM'} at 127.31.87.131:51329
I20260808 22:34:13.665122 32094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:34:13.673015 21221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:13.666999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38472 (local address 127.31.87.190:41657)
0808 22:34:13.667133 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:13.667137 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:13.667262 (+ 125us) server_negotiation.cc:407] Connection header received
0808 22:34:13.667360 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:13.667362 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:13.667398 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:13.667450 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:34:13.668013 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:13.668493 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:13.669284 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:13.669483 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:13.669554 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:13.670724 (+ 1170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:34:13.670727 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:34:13.670729 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:34:13.670743 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:34:13.671790 (+ 1047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:13.672134 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:13.672136 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:13.672138 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:13.672197 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:13.672374 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:13.672376 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:13.672377 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:13.672554 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:34:13.672649 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:13.672799 (+ 150us) server_negotiation.cc:299] Negotiation successful
0808 22:34:13.672880 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 22:34:13.675496 21159 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38472:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:34:13.678406 21159 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:34:13.691255 21906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:13.685963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:38117 (local address 127.0.0.1:45710)
0808 22:34:13.686430 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:13.686475 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 22:34:13.686602 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:34:13.687155 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:34:13.687160 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:34:13.687186 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:34:13.687321 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:13.687328 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:13.688599 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:13.688602 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:13.689668 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:13.689678 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:13.690731 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:13.690735 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:13.690845 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:13.690886 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 22:34:13.690994 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 22:34:13.691120 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":62,"threads_started":1}
I20260808 22:34:13.691301 21908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:13.686083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45710 (local address 127.31.87.129:38117)
0808 22:34:13.686455 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:13.686471 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:13.686544 (+ 73us) server_negotiation.cc:407] Connection header received
0808 22:34:13.686771 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:13.686778 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:13.686989 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:13.687156 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:34:13.687459 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:13.688489 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:13.689792 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:13.690609 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:13.690731 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:13.690878 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:13.691049 (+ 171us) server_negotiation.cc:299] Negotiation successful
0808 22:34:13.691145 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":95,"threads_started":1}
I20260808 22:34:13.691947 21909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:13.686107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.130:41469 (local address 127.0.0.1:58418)
0808 22:34:13.687064 (+ 957us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:13.687077 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:34:13.687156 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:34:13.687838 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:34:13.687841 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:34:13.687860 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:34:13.687949 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:13.687955 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:13.689241 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:13.689244 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:13.690241 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:13.690250 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:13.691435 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:13.691439 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:13.691683 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:13.691703 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 22:34:13.691762 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 22:34:13.691827 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":896,"thread_start_us":88,"threads_started":1}
I20260808 22:34:13.692046 21911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:13.686684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58418 (local address 127.31.87.130:41469)
0808 22:34:13.687020 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:13.687032 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:13.687240 (+ 208us) server_negotiation.cc:407] Connection header received
0808 22:34:13.687340 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:13.687345 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:13.687532 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:13.687670 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:34:13.688064 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:13.689113 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:13.690372 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:13.691145 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:13.691238 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:13.691370 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:13.691814 (+ 444us) server_negotiation.cc:299] Negotiation successful
0808 22:34:13.691908 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":125,"threads_started":1}
I20260808 22:34:13.692549 21910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:13.686612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.131:44849 (local address 127.0.0.1:56336)
0808 22:34:13.687418 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:13.687430 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:34:13.687563 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:34:13.688153 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:34:13.688156 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:34:13.688172 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:34:13.688254 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:13.688259 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:13.689535 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:13.689537 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:13.690623 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:13.690633 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:13.692008 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:13.692099 (+ 91us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:13.692254 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:13.692279 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 22:34:13.692360 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 22:34:13.692429 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":748,"thread_start_us":83,"threads_started":1}
I20260808 22:34:13.692678 21912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:13.686902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56336 (local address 127.31.87.131:44849)
0808 22:34:13.687272 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:13.687283 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:13.687499 (+ 216us) server_negotiation.cc:407] Connection header received
0808 22:34:13.687700 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:13.687706 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:13.687875 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:13.687983 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:34:13.688367 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:13.689418 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:13.690766 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:13.691697 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:13.691788 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:13.691912 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:13.692405 (+ 493us) server_negotiation.cc:299] Negotiation successful
0808 22:34:13.692503 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":90,"threads_started":1}
I20260808 22:34:13.694631 21363 tablet_service.cc:1467] Processing CreateTablet for tablet f660b76f57a54336875ecc812448c13f (DEFAULT_TABLE table=test-table [id=88e02209fe204d82bb63f933324ed529]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:34:13.694798 21576 tablet_service.cc:1467] Processing CreateTablet for tablet f660b76f57a54336875ecc812448c13f (DEFAULT_TABLE table=test-table [id=88e02209fe204d82bb63f933324ed529]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:34:13.695011 21777 tablet_service.cc:1467] Processing CreateTablet for tablet f660b76f57a54336875ecc812448c13f (DEFAULT_TABLE table=test-table [id=88e02209fe204d82bb63f933324ed529]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:34:13.695744 21363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f660b76f57a54336875ecc812448c13f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:34:13.695744 21576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f660b76f57a54336875ecc812448c13f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:34:13.695979 21777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f660b76f57a54336875ecc812448c13f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:34:13.702242 21913 tablet_bootstrap.cc:492] T f660b76f57a54336875ecc812448c13f P d686840dc3584fbd9b3c7accdb6e26a3: Bootstrap starting.
I20260808 22:34:13.703289 21914 tablet_bootstrap.cc:492] T f660b76f57a54336875ecc812448c13f P 4355aa00a45b496793ff2602e299e94e: Bootstrap starting.
I20260808 22:34:13.703651 21916 tablet_bootstrap.cc:492] T f660b76f57a54336875ecc812448c13f P 0db1fc6a077a4abb8d1650a2b9540d75: Bootstrap starting.
I20260808 22:34:13.704464 21913 tablet_bootstrap.cc:654] T f660b76f57a54336875ecc812448c13f P d686840dc3584fbd9b3c7accdb6e26a3: Neither blocks nor log segments found. Creating new log.
I20260808 22:34:13.705700 21913 log.cc:824] T f660b76f57a54336875ecc812448c13f P d686840dc3584fbd9b3c7accdb6e26a3: Log is configured to *not* fsync() on all Append() calls
I20260808 22:34:13.705907 21916 tablet_bootstrap.cc:654] T f660b76f57a54336875ecc812448c13f P 0db1fc6a077a4abb8d1650a2b9540d75: Neither blocks nor log segments found. Creating new log.
I20260808 22:34:13.706705 21916 log.cc:824] T f660b76f57a54336875ecc812448c13f P 0db1fc6a077a4abb8d1650a2b9540d75: Log is configured to *not* fsync() on all Append() calls
I20260808 22:34:13.707063 21914 tablet_bootstrap.cc:654] T f660b76f57a54336875ecc812448c13f P 4355aa00a45b496793ff2602e299e94e: Neither blocks nor log segments found. Creating new log.
I20260808 22:34:13.707859 21913 tablet_bootstrap.cc:492] T f660b76f57a54336875ecc812448c13f P d686840dc3584fbd9b3c7accdb6e26a3: No bootstrap required, opened a new log
I20260808 22:34:13.708051 21913 ts_tablet_manager.cc:1397] T f660b76f57a54336875ecc812448c13f P d686840dc3584fbd9b3c7accdb6e26a3: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 22:34:13.708307 21916 tablet_bootstrap.cc:492] T f660b76f57a54336875ecc812448c13f P 0db1fc6a077a4abb8d1650a2b9540d75: No bootstrap required, opened a new log
I20260808 22:34:13.708294 21914 log.cc:824] T f660b76f57a54336875ecc812448c13f P 4355aa00a45b496793ff2602e299e94e: Log is configured to *not* fsync() on all Append() calls
I20260808 22:34:13.708480 21916 ts_tablet_manager.cc:1397] T f660b76f57a54336875ecc812448c13f P 0db1fc6a077a4abb8d1650a2b9540d75: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 22:34:13.710182 21914 tablet_bootstrap.cc:492] T f660b76f57a54336875ecc812448c13f P 4355aa00a45b496793ff2602e299e94e: No bootstrap required, opened a new log
I20260808 22:34:13.710417 21914 ts_tablet_manager.cc:1397] T f660b76f57a54336875ecc812448c13f P 4355aa00a45b496793ff2602e299e94e: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.001s
I20260808 22:34:13.711159 21913 raft_consensus.cc:348] T f660b76f57a54336875ecc812448c13f P d686840dc3584fbd9b3c7accdb6e26a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4355aa00a45b496793ff2602e299e94e" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 38117 } } peers { permanent_uuid: "0db1fc6a077a4abb8d1650a2b9540d75" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41469 } } peers { permanent_uuid: "d686840dc3584fbd9b3c7accdb6e26a3" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 44849 } }
I20260808 22:34:13.711359 21913 raft_consensus.cc:374] T f660b76f57a54336875ecc812448c13f P d686840dc3584fbd9b3c7accdb6e26a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:34:13.711427 21913 raft_consensus.cc:729] T f660b76f57a54336875ecc812448c13f P d686840dc3584fbd9b3c7accdb6e26a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d686840dc3584fbd9b3c7accdb6e26a3, State: Initialized, Role: FOLLOWER
I20260808 22:34:13.711831 21913 consensus_queue.cc:260] T f660b76f57a54336875ecc812448c13f P d686840dc3584fbd9b3c7accdb6e26a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4355aa00a45b496793ff2602e299e94e" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 38117 } } peers { permanent_uuid: "0db1fc6a077a4abb8d1650a2b9540d75" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41469 } } peers { permanent_uuid: "d686840dc3584fbd9b3c7accdb6e26a3" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 44849 } }
I20260808 22:34:13.712106 21916 raft_consensus.cc:348] T f660b76f57a54336875ecc812448c13f P 0db1fc6a077a4abb8d1650a2b9540d75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4355aa00a45b496793ff2602e299e94e" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 38117 } } peers { permanent_uuid: "0db1fc6a077a4abb8d1650a2b9540d75" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41469 } } peers { permanent_uuid: "d686840dc3584fbd9b3c7accdb6e26a3" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 44849 } }
I20260808 22:34:13.712400 21889 heartbeater.cc:503] Master 127.31.87.190:41657 was elected leader, sending a full tablet report...
I20260808 22:34:13.712407 21916 raft_consensus.cc:374] T f660b76f57a54336875ecc812448c13f P 0db1fc6a077a4abb8d1650a2b9540d75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:34:13.712523 21916 raft_consensus.cc:729] T f660b76f57a54336875ecc812448c13f P 0db1fc6a077a4abb8d1650a2b9540d75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0db1fc6a077a4abb8d1650a2b9540d75, State: Initialized, Role: FOLLOWER
I20260808 22:34:13.712764 21913 ts_tablet_manager.cc:1428] T f660b76f57a54336875ecc812448c13f P d686840dc3584fbd9b3c7accdb6e26a3: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 22:34:13.713003 21916 consensus_queue.cc:260] T f660b76f57a54336875ecc812448c13f P 0db1fc6a077a4abb8d1650a2b9540d75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4355aa00a45b496793ff2602e299e94e" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 38117 } } peers { permanent_uuid: "0db1fc6a077a4abb8d1650a2b9540d75" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41469 } } peers { permanent_uuid: "d686840dc3584fbd9b3c7accdb6e26a3" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 44849 } }
I20260808 22:34:13.714013 21916 ts_tablet_manager.cc:1428] T f660b76f57a54336875ecc812448c13f P 0db1fc6a077a4abb8d1650a2b9540d75: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 22:34:13.714026 21914 raft_consensus.cc:348] T f660b76f57a54336875ecc812448c13f P 4355aa00a45b496793ff2602e299e94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4355aa00a45b496793ff2602e299e94e" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 38117 } } peers { permanent_uuid: "0db1fc6a077a4abb8d1650a2b9540d75" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41469 } } peers { permanent_uuid: "d686840dc3584fbd9b3c7accdb6e26a3" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 44849 } }
I20260808 22:34:13.714252 21914 raft_consensus.cc:374] T f660b76f57a54336875ecc812448c13f P 4355aa00a45b496793ff2602e299e94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:34:13.714319 21914 raft_consensus.cc:729] T f660b76f57a54336875ecc812448c13f P 4355aa00a45b496793ff2602e299e94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4355aa00a45b496793ff2602e299e94e, State: Initialized, Role: FOLLOWER
I20260808 22:34:13.714651 21914 consensus_queue.cc:260] T f660b76f57a54336875ecc812448c13f P 4355aa00a45b496793ff2602e299e94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4355aa00a45b496793ff2602e299e94e" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 38117 } } peers { permanent_uuid: "0db1fc6a077a4abb8d1650a2b9540d75" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41469 } } peers { permanent_uuid: "d686840dc3584fbd9b3c7accdb6e26a3" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 44849 } }
I20260808 22:34:13.715166 21668 heartbeater.cc:503] Master 127.31.87.190:41657 was elected leader, sending a full tablet report...
I20260808 22:34:13.715395 21469 heartbeater.cc:503] Master 127.31.87.190:41657 was elected leader, sending a full tablet report...
I20260808 22:34:13.715497 21914 ts_tablet_manager.cc:1428] T f660b76f57a54336875ecc812448c13f P 4355aa00a45b496793ff2602e299e94e: Time spent starting tablet: real 0.005s user 0.001s sys 0.003s
W20260808 22:34:13.719907 21472 tablet.cc:2369] T f660b76f57a54336875ecc812448c13f P 4355aa00a45b496793ff2602e299e94e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 22:34:13.898566 21890 tablet.cc:2369] T f660b76f57a54336875ecc812448c13f P d686840dc3584fbd9b3c7accdb6e26a3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 22:34:13.924381 21669 tablet.cc:2369] T f660b76f57a54336875ecc812448c13f P 0db1fc6a077a4abb8d1650a2b9540d75: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:34:14.042716 21924 raft_consensus.cc:482] T f660b76f57a54336875ecc812448c13f P 4355aa00a45b496793ff2602e299e94e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:34:14.042968 21924 raft_consensus.cc:504] T f660b76f57a54336875ecc812448c13f P 4355aa00a45b496793ff2602e299e94e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4355aa00a45b496793ff2602e299e94e" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 38117 } } peers { permanent_uuid: "0db1fc6a077a4abb8d1650a2b9540d75" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41469 } } peers { permanent_uuid: "d686840dc3584fbd9b3c7accdb6e26a3" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 44849 } }
I20260808 22:34:14.044230 21924 leader_election.cc:288] T f660b76f57a54336875ecc812448c13f P 4355aa00a45b496793ff2602e299e94e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0db1fc6a077a4abb8d1650a2b9540d75 (127.31.87.130:41469), d686840dc3584fbd9b3c7accdb6e26a3 (127.31.87.131:44849)
I20260808 22:34:14.048035 21912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:14.044790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:48373 (local address 127.31.87.131:44849)
0808 22:34:14.044913 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:14.044917 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:14.044932 (+ 15us) server_negotiation.cc:407] Connection header received
0808 22:34:14.045081 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:14.045103 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:14.045153 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:14.045220 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:34:14.045485 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:14.045983 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:14.047117 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:14.047501 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:14.047611 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:14.047662 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:14.047845 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 22:34:14.047914 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 22:34:14.048048 21911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:14.044672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:39171 (local address 127.31.87.130:41469)
0808 22:34:14.044797 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:14.044802 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:14.044880 (+ 78us) server_negotiation.cc:407] Connection header received
0808 22:34:14.045138 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:14.045141 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:14.045187 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:14.045254 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:34:14.045520 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:14.045991 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:14.047100 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:14.047489 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:14.047594 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:14.047650 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:14.047853 (+ 203us) server_negotiation.cc:299] Negotiation successful
0808 22:34:14.047922 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 22:34:14.048077 21925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:14.044450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.130:41469 (local address 127.31.87.129:39171)
0808 22:34:14.044795 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:14.044809 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:34:14.044976 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:34:14.045276 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:34:14.045279 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:34:14.045315 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:34:14.045416 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:14.045421 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:14.046096 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:14.046099 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:14.046979 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:14.046988 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:14.047596 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:14.047599 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:14.047720 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:14.047754 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 22:34:14.047826 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 22:34:14.047921 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"mutex_wait_us":75,"spinlock_wait_cycles":14592,"thread_start_us":116,"threads_started":1}
I20260808 22:34:14.048087 21927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:14.044463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.131:44849 (local address 127.31.87.129:48373)
0808 22:34:14.044831 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:14.044845 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:34:14.044964 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:34:14.045251 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:34:14.045254 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:34:14.045270 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:34:14.045386 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:14.045391 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:14.046091 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:14.046095 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:14.047001 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:14.047011 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:14.047610 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:14.047613 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:14.047722 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:14.047756 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 22:34:14.047821 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 22:34:14.047904 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"mutex_wait_us":77,"spinlock_wait_cycles":17024,"thread_start_us":94,"threads_started":1}
I20260808 22:34:14.048782 21801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f660b76f57a54336875ecc812448c13f" candidate_uuid: "4355aa00a45b496793ff2602e299e94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d686840dc3584fbd9b3c7accdb6e26a3" is_pre_election: true
I20260808 22:34:14.048982 21600 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f660b76f57a54336875ecc812448c13f" candidate_uuid: "4355aa00a45b496793ff2602e299e94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0db1fc6a077a4abb8d1650a2b9540d75" is_pre_election: true
I20260808 22:34:14.049047 21801 raft_consensus.cc:2454] T f660b76f57a54336875ecc812448c13f P d686840dc3584fbd9b3c7accdb6e26a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4355aa00a45b496793ff2602e299e94e in term 0.
I20260808 22:34:14.049273 21600 raft_consensus.cc:2454] T f660b76f57a54336875ecc812448c13f P 0db1fc6a077a4abb8d1650a2b9540d75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4355aa00a45b496793ff2602e299e94e in term 0.
I20260808 22:34:14.049542 21290 leader_election.cc:302] T f660b76f57a54336875ecc812448c13f P 4355aa00a45b496793ff2602e299e94e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4355aa00a45b496793ff2602e299e94e, d686840dc3584fbd9b3c7accdb6e26a3; no voters:
I20260808 22:34:14.049798 21924 raft_consensus.cc:2781] T f660b76f57a54336875ecc812448c13f P 4355aa00a45b496793ff2602e299e94e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:34:14.049873 21924 raft_consensus.cc:482] T f660b76f57a54336875ecc812448c13f P 4355aa00a45b496793ff2602e299e94e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:34:14.049904 21924 raft_consensus.cc:3037] T f660b76f57a54336875ecc812448c13f P 4355aa00a45b496793ff2602e299e94e [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:34:14.050908 21924 raft_consensus.cc:504] T f660b76f57a54336875ecc812448c13f P 4355aa00a45b496793ff2602e299e94e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4355aa00a45b496793ff2602e299e94e" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 38117 } } peers { permanent_uuid: "0db1fc6a077a4abb8d1650a2b9540d75" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41469 } } peers { permanent_uuid: "d686840dc3584fbd9b3c7accdb6e26a3" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 44849 } }
I20260808 22:34:14.051236 21924 leader_election.cc:288] T f660b76f57a54336875ecc812448c13f P 4355aa00a45b496793ff2602e299e94e [CANDIDATE]: Term 1 election: Requested vote from peers 0db1fc6a077a4abb8d1650a2b9540d75 (127.31.87.130:41469), d686840dc3584fbd9b3c7accdb6e26a3 (127.31.87.131:44849)
I20260808 22:34:14.051573 21600 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f660b76f57a54336875ecc812448c13f" candidate_uuid: "4355aa00a45b496793ff2602e299e94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0db1fc6a077a4abb8d1650a2b9540d75"
I20260808 22:34:14.051573 21801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f660b76f57a54336875ecc812448c13f" candidate_uuid: "4355aa00a45b496793ff2602e299e94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d686840dc3584fbd9b3c7accdb6e26a3"
I20260808 22:34:14.051689 21600 raft_consensus.cc:3037] T f660b76f57a54336875ecc812448c13f P 0db1fc6a077a4abb8d1650a2b9540d75 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:34:14.051689 21801 raft_consensus.cc:3037] T f660b76f57a54336875ecc812448c13f P d686840dc3584fbd9b3c7accdb6e26a3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:34:14.052870 21600 raft_consensus.cc:2454] T f660b76f57a54336875ecc812448c13f P 0db1fc6a077a4abb8d1650a2b9540d75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4355aa00a45b496793ff2602e299e94e in term 1.
I20260808 22:34:14.052870 21801 raft_consensus.cc:2454] T f660b76f57a54336875ecc812448c13f P d686840dc3584fbd9b3c7accdb6e26a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4355aa00a45b496793ff2602e299e94e in term 1.
I20260808 22:34:14.053160 21288 leader_election.cc:302] T f660b76f57a54336875ecc812448c13f P 4355aa00a45b496793ff2602e299e94e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0db1fc6a077a4abb8d1650a2b9540d75, 4355aa00a45b496793ff2602e299e94e; no voters:
I20260808 22:34:14.053306 21924 raft_consensus.cc:2781] T f660b76f57a54336875ecc812448c13f P 4355aa00a45b496793ff2602e299e94e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:34:14.053504 21924 raft_consensus.cc:686] T f660b76f57a54336875ecc812448c13f P 4355aa00a45b496793ff2602e299e94e [term 1 LEADER]: Becoming Leader. State: Replica: 4355aa00a45b496793ff2602e299e94e, State: Running, Role: LEADER
I20260808 22:34:14.053802 21924 consensus_queue.cc:237] T f660b76f57a54336875ecc812448c13f P 4355aa00a45b496793ff2602e299e94e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4355aa00a45b496793ff2602e299e94e" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 38117 } } peers { permanent_uuid: "0db1fc6a077a4abb8d1650a2b9540d75" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41469 } } peers { permanent_uuid: "d686840dc3584fbd9b3c7accdb6e26a3" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 44849 } }
I20260808 22:34:14.056931 21154 catalog_manager.cc:5495] T f660b76f57a54336875ecc812448c13f P 4355aa00a45b496793ff2602e299e94e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4355aa00a45b496793ff2602e299e94e (127.31.87.129). New cstate: current_term: 1 leader_uuid: "4355aa00a45b496793ff2602e299e94e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4355aa00a45b496793ff2602e299e94e" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 38117 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0db1fc6a077a4abb8d1650a2b9540d75" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d686840dc3584fbd9b3c7accdb6e26a3" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 44849 } health_report { overall_health: UNKNOWN } } }
I20260808 22:34:14.140636 21908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:14.137658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45716 (local address 127.31.87.129:38117)
0808 22:34:14.137777 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:14.137781 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:14.137794 (+ 13us) server_negotiation.cc:407] Connection header received
0808 22:34:14.137894 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:14.137896 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:14.137953 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:14.138040 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:34:14.138238 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:14.138713 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:14.139477 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:14.139675 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:14.139738 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:14.139975 (+ 237us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:34:14.140357 (+ 382us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:34:14.140439 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:14.140484 (+ 45us) server_negotiation.cc:299] Negotiation successful
0808 22:34:14.140530 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 22:34:14.146286 21801 raft_consensus.cc:1261] T f660b76f57a54336875ecc812448c13f P d686840dc3584fbd9b3c7accdb6e26a3 [term 1 FOLLOWER]: Refusing update from remote peer 4355aa00a45b496793ff2602e299e94e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:34:14.146286 21600 raft_consensus.cc:1261] T f660b76f57a54336875ecc812448c13f P 0db1fc6a077a4abb8d1650a2b9540d75 [term 1 FOLLOWER]: Refusing update from remote peer 4355aa00a45b496793ff2602e299e94e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:34:14.147037 21928 consensus_queue.cc:1035] T f660b76f57a54336875ecc812448c13f P 4355aa00a45b496793ff2602e299e94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0db1fc6a077a4abb8d1650a2b9540d75" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:34:14.147214 21924 consensus_queue.cc:1035] T f660b76f57a54336875ecc812448c13f P 4355aa00a45b496793ff2602e299e94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d686840dc3584fbd9b3c7accdb6e26a3" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 44849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:34:14.154074 21932 mvcc.cc:205] Tried to move back new op lower bound from 7316391748175736832 to 7316391747806760960. Current Snapshot: MvccSnapshot[applied={T|T < 7316391748175736832}]
I20260808 22:34:14.155205 21933 mvcc.cc:205] Tried to move back new op lower bound from 7316391748175736832 to 7316391747806760960. Current Snapshot: MvccSnapshot[applied={T|T < 7316391748175736832}]
I20260808 22:34:14.164404 21157 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38472:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:34:14.164541 21157 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38472:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:34:14.166823 21157 catalog_manager.cc:5782] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439: Sending DeleteTablet for 3 replicas of tablet f660b76f57a54336875ecc812448c13f
I20260808 22:34:14.167550 21363 tablet_service.cc:1514] Processing DeleteTablet for tablet f660b76f57a54336875ecc812448c13f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:34:14 UTC) from {username='kudu', principal='kudu/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:45710
I20260808 22:34:14.167553 21576 tablet_service.cc:1514] Processing DeleteTablet for tablet f660b76f57a54336875ecc812448c13f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:34:14 UTC) from {username='kudu', principal='kudu/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:58418
I20260808 22:34:14.167660 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 21258
I20260808 22:34:14.177258 21777 tablet_service.cc:1514] Processing DeleteTablet for tablet f660b76f57a54336875ecc812448c13f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:34:14 UTC) from {username='kudu', principal='kudu/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:56336
I20260808 22:34:14.177356 21943 tablet_replica.cc:330] T f660b76f57a54336875ecc812448c13f P 0db1fc6a077a4abb8d1650a2b9540d75: stopping tablet replica
I20260808 22:34:14.177642 21943 raft_consensus.cc:2229] T f660b76f57a54336875ecc812448c13f P 0db1fc6a077a4abb8d1650a2b9540d75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:34:14.177907 21943 raft_consensus.cc:2258] T f660b76f57a54336875ecc812448c13f P 0db1fc6a077a4abb8d1650a2b9540d75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:34:14.179335 21943 ts_tablet_manager.cc:1905] T f660b76f57a54336875ecc812448c13f P 0db1fc6a077a4abb8d1650a2b9540d75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:34:14.180135 21944 tablet_replica.cc:330] T f660b76f57a54336875ecc812448c13f P d686840dc3584fbd9b3c7accdb6e26a3: stopping tablet replica
I20260808 22:34:14.180374 21944 raft_consensus.cc:2229] T f660b76f57a54336875ecc812448c13f P d686840dc3584fbd9b3c7accdb6e26a3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:34:14.180627 21944 raft_consensus.cc:2258] T f660b76f57a54336875ecc812448c13f P d686840dc3584fbd9b3c7accdb6e26a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:34:14.181838 21944 ts_tablet_manager.cc:1905] T f660b76f57a54336875ecc812448c13f P d686840dc3584fbd9b3c7accdb6e26a3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:34:14.182315 21943 ts_tablet_manager.cc:1918] T f660b76f57a54336875ecc812448c13f P 0db1fc6a077a4abb8d1650a2b9540d75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:34:14.182430 21943 log.cc:1196] T f660b76f57a54336875ecc812448c13f P 0db1fc6a077a4abb8d1650a2b9540d75: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-1/wal/wals/f660b76f57a54336875ecc812448c13f
I20260808 22:34:14.182821 21943 ts_tablet_manager.cc:1939] T f660b76f57a54336875ecc812448c13f P 0db1fc6a077a4abb8d1650a2b9540d75: Deleting consensus metadata
I20260808 22:34:14.184032 21944 ts_tablet_manager.cc:1918] T f660b76f57a54336875ecc812448c13f P d686840dc3584fbd9b3c7accdb6e26a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:34:14.184058 21135 catalog_manager.cc:4841] TS 0db1fc6a077a4abb8d1650a2b9540d75 (127.31.87.130:41469): tablet f660b76f57a54336875ecc812448c13f (table test-table [id=88e02209fe204d82bb63f933324ed529]) successfully deleted
I20260808 22:34:14.184135 21944 log.cc:1196] T f660b76f57a54336875ecc812448c13f P d686840dc3584fbd9b3c7accdb6e26a3: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-2/wal/wals/f660b76f57a54336875ecc812448c13f
I20260808 22:34:14.184398 21944 ts_tablet_manager.cc:1939] T f660b76f57a54336875ecc812448c13f P d686840dc3584fbd9b3c7accdb6e26a3: Deleting consensus metadata
I20260808 22:34:14.185348 21136 catalog_manager.cc:4841] TS d686840dc3584fbd9b3c7accdb6e26a3 (127.31.87.131:44849): tablet f660b76f57a54336875ecc812448c13f (table test-table [id=88e02209fe204d82bb63f933324ed529]) successfully deleted
W20260808 22:34:14.187263 21136 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.87.129:38117 (error 108)
I20260808 22:34:14.187541 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 21478
I20260808 22:34:14.190438 21910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:14.190077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:38117 (local address 127.0.0.1:45722)
0808 22:34:14.190249 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:14.190339 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.129:38117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
W20260808 22:34:14.190760 21136 catalog_manager.cc:4584] TS 4355aa00a45b496793ff2602e299e94e (127.31.87.129:38117): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f660b76f57a54336875ecc812448c13f: Network error: Client connection negotiation failed: client connection to 127.31.87.129:38117: connect: Connection refused (error 111)
I20260808 22:34:14.197801 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 21673
I20260808 22:34:14.207408 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 21109
I20260808 22:34:14.218371 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:41657
--webserver_interface=127.31.87.190
--webserver_port=37393
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:43005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:41657
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:34:14.325134 21947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:34:14.325363 21947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:34:14.325418 21947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:34:14.329257 21947 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:34:14.329301 21947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:34:14.329327 21947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:34:14.329348 21947 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:34:14.329377 21947 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:34:14.334331 21947 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:43005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:41657
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:41657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.87.190
--webserver_port=37393
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21947
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:34:14.335216 21947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:34:14.336102 21947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:34:14.348857 21956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:34:14.348857 21955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:34:14.349293 21960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:34:14.349654 21947 server_base.cc:1034] running on GCE node
I20260808 22:34:14.350207 21947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:34:14.351305 21947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:34:14.352459 21947 hybrid_clock.cc:648] HybridClock initialized: now 1786228454352441 us; error 28 us; skew 500 ppm
Aug 08 22:34:14 dist-test-slave-266d krb5kdc[21091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228454, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:34:14.355124 21947 init.cc:377] Logged in from keytab as kudu/127.31.87.190@KRBTEST.COM (short username kudu)
I20260808 22:34:14.355983 21947 webserver.cc:460] Webserver started at http://127.31.87.190:37393/ using document root <none> and password file <none>
I20260808 22:34:14.356562 21947 fs_manager.cc:359] Metadata directory not provided
I20260808 22:34:14.356642 21947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:34:14.360373 21947 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:34:14.362462 21973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:14.363602 21947 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:34:14.363723 21947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "aa7651551d8d498890f0984c21549439"
format_stamp: "Formatted at 2026-08-08 22:34:12 on dist-test-slave-266d"
server_key: "8c08cb5db44f911c46876007b3896cb8"
server_key_iv: "4ab580b9b3a9561b384123b0f8a3a859"
server_key_version: "encryptionkey@0"
I20260808 22:34:14.364305 21947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:34:14.384209 21947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:34:14.386765 21947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:34:14.386934 21947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:34:14.393407 21947 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.190:41657
I20260808 22:34:14.393651 22052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.190:41657 every 8 connection(s)
I20260808 22:34:14.394429 21947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:34:14.398512 22053 sys_catalog.cc:263] Verifying existing consensus state
I20260808 22:34:14.400036 22053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439: Bootstrap starting.
I20260808 22:34:14.404462 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 21947
I20260808 22:34:14.404958 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.129:38117
--local_ip_for_outbound_sockets=127.31.87.129
--tserver_master_addrs=127.31.87.190:41657
--webserver_port=34235
--webserver_interface=127.31.87.129
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab
--principal=kudu/127.31.87.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:43005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 22:34:14.416873 22053 log.cc:824] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439: Log is configured to *not* fsync() on all Append() calls
I20260808 22:34:14.425246 22053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 22:34:14.425546 22053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439: Bootstrap complete.
I20260808 22:34:14.429222 22053 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa7651551d8d498890f0984c21549439" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41657 } }
I20260808 22:34:14.429600 22053 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa7651551d8d498890f0984c21549439, State: Initialized, Role: FOLLOWER
I20260808 22:34:14.430042 22053 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa7651551d8d498890f0984c21549439" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41657 } }
I20260808 22:34:14.430150 22053 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:34:14.430215 22053 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:34:14.430305 22053 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439 [term 1 FOLLOWER]: Advancing to term 2
I20260808 22:34:14.432014 22053 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa7651551d8d498890f0984c21549439" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41657 } }
I20260808 22:34:14.432305 22053 leader_election.cc:302] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa7651551d8d498890f0984c21549439; no voters:
I20260808 22:34:14.432572 22053 leader_election.cc:288] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 22:34:14.432660 22059 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 22:34:14.432870 22059 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439 [term 2 LEADER]: Becoming Leader. State: Replica: aa7651551d8d498890f0984c21549439, State: Running, Role: LEADER
I20260808 22:34:14.433174 22059 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa7651551d8d498890f0984c21549439" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41657 } }
I20260808 22:34:14.433791 22053 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:34:14.434069 22059 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa7651551d8d498890f0984c21549439. Latest consensus state: current_term: 2 leader_uuid: "aa7651551d8d498890f0984c21549439" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa7651551d8d498890f0984c21549439" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41657 } } }
I20260808 22:34:14.434270 22059 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439 [sys.catalog]: This master's current role is: LEADER
I20260808 22:34:14.435336 22060 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "aa7651551d8d498890f0984c21549439" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa7651551d8d498890f0984c21549439" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41657 } } }
I20260808 22:34:14.435447 22060 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439 [sys.catalog]: This master's current role is: LEADER
I20260808 22:34:14.435322 22065 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:34:14.439139 22065 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:34:14.439621 22065 catalog_manager.cc:1259] Loaded cluster ID: 096d94a0830e4ccc8476217ffb722cc7
I20260808 22:34:14.439670 22065 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:34:14.440770 22065 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260808 22:34:14.443512 22065 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260808 22:34:14.447288 22065 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:34:14.447829 22065 catalog_manager.cc:5879] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439: Loaded TSK: 0
I20260808 22:34:14.448679 22065 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 22:34:14.525555 22056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:34:14.525784 22056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:34:14.525839 22056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:34:14.529700 22056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:34:14.529750 22056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:34:14.529822 22056 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.129
I20260808 22:34:14.534902 22056 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:43005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab
--principal=kudu/127.31.87.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.129:38117
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.87.129
--webserver_port=34235
--tserver_master_addrs=127.31.87.190:41657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22056
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.129
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:34:14.535813 22056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:34:14.536715 22056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:34:14.545584 22099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:34:14.545609 22097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:34:14.545675 22056 server_base.cc:1034] running on GCE node
W20260808 22:34:14.545608 22096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:34:14.546218 22056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:34:14.546795 22056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:34:14.547966 22056 hybrid_clock.cc:648] HybridClock initialized: now 1786228454547944 us; error 31 us; skew 500 ppm
Aug 08 22:34:14 dist-test-slave-266d krb5kdc[21091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228454, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:34:14.550587 22056 init.cc:377] Logged in from keytab as kudu/127.31.87.129@KRBTEST.COM (short username kudu)
I20260808 22:34:14.551431 22056 webserver.cc:460] Webserver started at http://127.31.87.129:34235/ using document root <none> and password file <none>
I20260808 22:34:14.551941 22056 fs_manager.cc:359] Metadata directory not provided
I20260808 22:34:14.552011 22056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:34:14.555708 22056 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:34:14.557859 22108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:14.559054 22056 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:34:14.559154 22056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-0/wal
uuid: "4355aa00a45b496793ff2602e299e94e"
format_stamp: "Formatted at 2026-08-08 22:34:13 on dist-test-slave-266d"
server_key: "fcb44dde4838b861fb64d9a0e7ef8919"
server_key_iv: "8a0b22cc07e34d0e72cb43451eaa5325"
server_key_version: "encryptionkey@0"
I20260808 22:34:14.559629 22056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:34:14.589752 22056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:34:14.592324 22056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:34:14.592491 22056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:34:14.593048 22056 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:34:14.594091 22120 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 22:34:14.598104 22056 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 22:34:14.598155 22056 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260808 22:34:14.598215 22056 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 22:34:14.599727 22056 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 22:34:14.599779 22056 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260808 22:34:14.600001 22120 tablet_bootstrap.cc:492] T f660b76f57a54336875ecc812448c13f P 4355aa00a45b496793ff2602e299e94e: Bootstrap starting.
I20260808 22:34:14.608657 22056 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.129:38117
I20260808 22:34:14.608820 22285 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.129:38117 every 8 connection(s)
I20260808 22:34:14.609591 22056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
I20260808 22:34:14.612145 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 22056
I20260808 22:34:14.612541 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.130:41469
--local_ip_for_outbound_sockets=127.31.87.130
--tserver_master_addrs=127.31.87.190:41657
--webserver_port=44951
--webserver_interface=127.31.87.130
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab
--principal=kudu/127.31.87.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:43005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 22:34:14 dist-test-slave-266d krb5kdc[21091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228454, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.129@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:34:14.621016 22120 log.cc:824] T f660b76f57a54336875ecc812448c13f P 4355aa00a45b496793ff2602e299e94e: Log is configured to *not* fsync() on all Append() calls
I20260808 22:34:14.626642 22288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:14.611334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.129:33141)
0808 22:34:14.611773 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:14.611821 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 22:34:14.613982 (+ 2161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:34:14.615125 (+ 1143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:34:14.615133 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:34:14.615514 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:34:14.615937 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:14.615950 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:14.617209 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:14.617213 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:14.618032 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:14.618042 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:14.618763 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:14.618768 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:14.618954 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:14.619543 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:34:14.619559 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:34:14.620901 (+ 1342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:34:14.623347 (+ 2446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:14.623353 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:14.623366 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:14.623679 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:14.624076 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:14.624079 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:14.624081 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:14.624246 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:14.624965 (+ 719us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:34:14.624970 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:34:14.625253 (+ 283us) client_negotiation.cc:769] Sending connection context
0808 22:34:14.625442 (+ 189us) client_negotiation.cc:240] Negotiation successful
0808 22:34:14.625736 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":112,"threads_started":1}
I20260808 22:34:14.626734 22289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:14.611394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:33141 (local address 127.31.87.190:41657)
0808 22:34:14.611855 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:14.611874 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:14.611920 (+ 46us) server_negotiation.cc:407] Connection header received
0808 22:34:14.614485 (+ 2565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:14.614496 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:14.614774 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:14.615053 (+ 279us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:34:14.616094 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:14.617045 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:14.618146 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:14.618634 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:14.618803 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:14.621071 (+ 2268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:34:14.621076 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:34:14.621109 (+ 33us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:34:14.621142 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:34:14.623218 (+ 2076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:14.623800 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:14.623806 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:14.623812 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:14.623969 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:14.624355 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:14.624359 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:14.624360 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:14.624840 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:34:14.624982 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:14.625624 (+ 642us) server_negotiation.cc:299] Negotiation successful
0808 22:34:14.625931 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":113,"threads_started":1}
I20260808 22:34:14.628981 22286 heartbeater.cc:347] Connected to a master server at 127.31.87.190:41657
I20260808 22:34:14.628979 22120 tablet_bootstrap.cc:492] T f660b76f57a54336875ecc812448c13f P 4355aa00a45b496793ff2602e299e94e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 22:34:14.629271 22286 heartbeater.cc:464] Registering TS with master...
I20260808 22:34:14.629410 22120 tablet_bootstrap.cc:492] T f660b76f57a54336875ecc812448c13f P 4355aa00a45b496793ff2602e299e94e: Bootstrap complete.
I20260808 22:34:14.629778 22286 heartbeater.cc:511] Master 127.31.87.190:41657 requested a full tablet report, sending...
I20260808 22:34:14.630379 22120 ts_tablet_manager.cc:1397] T f660b76f57a54336875ecc812448c13f P 4355aa00a45b496793ff2602e299e94e: Time spent bootstrapping tablet: real 0.031s user 0.023s sys 0.004s
I20260808 22:34:14.632361 21997 ts_manager.cc:194] Registered new tserver with Master: 4355aa00a45b496793ff2602e299e94e (127.31.87.129:38117)
I20260808 22:34:14.632856 22120 raft_consensus.cc:348] T f660b76f57a54336875ecc812448c13f P 4355aa00a45b496793ff2602e299e94e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4355aa00a45b496793ff2602e299e94e" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 38117 } } peers { permanent_uuid: "0db1fc6a077a4abb8d1650a2b9540d75" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41469 } } peers { permanent_uuid: "d686840dc3584fbd9b3c7accdb6e26a3" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 44849 } }
I20260808 22:34:14.633436 22120 raft_consensus.cc:729] T f660b76f57a54336875ecc812448c13f P 4355aa00a45b496793ff2602e299e94e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4355aa00a45b496793ff2602e299e94e, State: Initialized, Role: FOLLOWER
I20260808 22:34:14.633905 22120 consensus_queue.cc:260] T f660b76f57a54336875ecc812448c13f P 4355aa00a45b496793ff2602e299e94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4355aa00a45b496793ff2602e299e94e" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 38117 } } peers { permanent_uuid: "0db1fc6a077a4abb8d1650a2b9540d75" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41469 } } peers { permanent_uuid: "d686840dc3584fbd9b3c7accdb6e26a3" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 44849 } }
I20260808 22:34:14.634954 22120 ts_tablet_manager.cc:1428] T f660b76f57a54336875ecc812448c13f P 4355aa00a45b496793ff2602e299e94e: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 22:34:14.636019 21997 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.129@KRBTEST.COM'} at 127.31.87.129:33141
I20260808 22:34:14.638345 22286 heartbeater.cc:503] Master 127.31.87.190:41657 was elected leader, sending a full tablet report...
Aug 08 22:34:14 dist-test-slave-266d krb5kdc[21091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228454, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for kudu/127.31.87.129@KRBTEST.COM
I20260808 22:34:14.644944 22293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:14.635490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:38117 (local address 127.0.0.1:45728)
0808 22:34:14.635867 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:14.635909 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 22:34:14.636027 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:34:14.636786 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:34:14.636792 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:34:14.637183 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:34:14.637315 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:14.637323 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:14.638803 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:14.638806 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:14.639490 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:14.639500 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:14.640012 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:14.640016 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:14.640128 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:14.640586 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:34:14.640597 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:34:14.641814 (+ 1217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:34:14.643444 (+ 1630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:14.643453 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:14.643458 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:14.643656 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:14.643922 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:14.643925 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:14.643927 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:14.644002 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:14.644389 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:34:14.644395 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:34:14.644500 (+ 105us) client_negotiation.cc:769] Sending connection context
0808 22:34:14.644610 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 22:34:14.644783 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":116,"threads_started":1}
I20260808 22:34:14.644975 22296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:14.636116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45728 (local address 127.31.87.129:38117)
0808 22:34:14.636396 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:14.636407 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:14.636428 (+ 21us) server_negotiation.cc:407] Connection header received
0808 22:34:14.636485 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:14.636490 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:14.636664 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:14.636796 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:34:14.637456 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:14.638579 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:14.639597 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:14.639912 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:14.640000 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:14.641933 (+ 1933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:34:14.641943 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:34:14.641955 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:34:14.641974 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:34:14.643351 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:14.643754 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:14.643760 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:14.643762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:14.643841 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:14.644077 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:14.644080 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:14.644082 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:14.644286 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:34:14.644393 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:14.644696 (+ 303us) server_negotiation.cc:299] Negotiation successful
0808 22:34:14.644843 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":89,"threads_started":1}
I20260808 22:34:14.645972 22190 tablet_service.cc:1514] Processing DeleteTablet for tablet f660b76f57a54336875ecc812448c13f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:34:14 UTC) from {username='kudu', principal='kudu/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:45728
I20260808 22:34:14.646651 22297 tablet_replica.cc:330] stopping tablet replica
I20260808 22:34:14.646889 22297 raft_consensus.cc:2229] T f660b76f57a54336875ecc812448c13f P 4355aa00a45b496793ff2602e299e94e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:34:14.647084 22297 raft_consensus.cc:2258] T f660b76f57a54336875ecc812448c13f P 4355aa00a45b496793ff2602e299e94e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:34:14.647850 22297 ts_tablet_manager.cc:1905] T f660b76f57a54336875ecc812448c13f P 4355aa00a45b496793ff2602e299e94e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:34:14.650996 22297 ts_tablet_manager.cc:1918] T f660b76f57a54336875ecc812448c13f P 4355aa00a45b496793ff2602e299e94e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:34:14.651070 22297 log.cc:1196] T f660b76f57a54336875ecc812448c13f P 4355aa00a45b496793ff2602e299e94e: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-0/wal/wals/f660b76f57a54336875ecc812448c13f
I20260808 22:34:14.651288 22297 ts_tablet_manager.cc:1939] T f660b76f57a54336875ecc812448c13f P 4355aa00a45b496793ff2602e299e94e: Deleting consensus metadata
I20260808 22:34:14.652612 21979 catalog_manager.cc:4841] TS 4355aa00a45b496793ff2602e299e94e (127.31.87.129:38117): tablet f660b76f57a54336875ecc812448c13f (table test-table [id=88e02209fe204d82bb63f933324ed529]) successfully deleted
W20260808 22:34:14.727571 22290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:34:14.727802 22290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:34:14.727856 22290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:34:14.731765 22290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:34:14.731819 22290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:34:14.731895 22290 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.130
I20260808 22:34:14.737048 22290 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:43005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab
--principal=kudu/127.31.87.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.130:41469
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.87.130
--webserver_port=44951
--tserver_master_addrs=127.31.87.190:41657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22290
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.130
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:34:14.737998 22290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:34:14.738933 22290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:34:14.748135 22304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:34:14.748164 22305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:34:14.748135 22307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:34:14.748193 22290 server_base.cc:1034] running on GCE node
I20260808 22:34:14.748777 22290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:34:14.749397 22290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:34:14.750571 22290 hybrid_clock.cc:648] HybridClock initialized: now 1786228454750544 us; error 39 us; skew 500 ppm
Aug 08 22:34:14 dist-test-slave-266d krb5kdc[21091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228454, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:34:14.753065 22290 init.cc:377] Logged in from keytab as kudu/127.31.87.130@KRBTEST.COM (short username kudu)
I20260808 22:34:14.753934 22290 webserver.cc:460] Webserver started at http://127.31.87.130:44951/ using document root <none> and password file <none>
I20260808 22:34:14.754456 22290 fs_manager.cc:359] Metadata directory not provided
I20260808 22:34:14.754537 22290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:34:14.758124 22290 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:34:14.759979 22317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:14.761054 22290 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:34:14.761188 22290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-1/wal
uuid: "0db1fc6a077a4abb8d1650a2b9540d75"
format_stamp: "Formatted at 2026-08-08 22:34:13 on dist-test-slave-266d"
server_key: "e06bdce1b696c3ead6da19debab55c73"
server_key_iv: "5cf8f2a82dbecb53ae41691fdad2c48f"
server_key_version: "encryptionkey@0"
I20260808 22:34:14.761669 22290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:34:14.773337 22290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:34:14.776124 22290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:34:14.776283 22290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:34:14.776800 22290 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:34:14.777581 22290 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:34:14.777630 22290 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:14.777678 22290 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:34:14.777707 22290 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:14.785943 22290 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.130:41469
I20260808 22:34:14.785972 22498 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.130:41469 every 8 connection(s)
I20260808 22:34:14.786996 22290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
I20260808 22:34:14.790926 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 22290
I20260808 22:34:14.791298 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.131:44849
--local_ip_for_outbound_sockets=127.31.87.131
--tserver_master_addrs=127.31.87.190:41657
--webserver_port=42581
--webserver_interface=127.31.87.131
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab
--principal=kudu/127.31.87.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:43005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 22:34:14 dist-test-slave-266d krb5kdc[21091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228454, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.130@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:34:14.799605 22289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:14.788646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:35447 (local address 127.31.87.190:41657)
0808 22:34:14.788770 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:14.788773 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:14.789544 (+ 771us) server_negotiation.cc:407] Connection header received
0808 22:34:14.790823 (+ 1279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:14.790827 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:14.790891 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:14.790966 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:34:14.792120 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:14.792650 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:14.793789 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:14.794032 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:14.794123 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:14.796392 (+ 2269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:34:14.796395 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:34:14.796397 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:34:14.796421 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:34:14.797732 (+ 1311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:14.798200 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:14.798203 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:14.798204 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:14.798280 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:14.798581 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:14.798585 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:14.798586 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:14.798761 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:34:14.798848 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:14.799309 (+ 461us) server_negotiation.cc:299] Negotiation successful
0808 22:34:14.799443 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 22:34:14.800323 22502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:14.788974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.130:35447)
0808 22:34:14.789404 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:14.789443 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 22:34:14.790619 (+ 1176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:34:14.791146 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:34:14.791153 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:34:14.791519 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:34:14.791974 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:14.791988 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:14.792771 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:14.792774 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:14.793665 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:14.793675 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:14.794176 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:14.794180 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:14.794335 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:14.794845 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:34:14.794861 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:34:14.796224 (+ 1363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:34:14.797840 (+ 1616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:14.797846 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:14.797859 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:14.798106 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:14.798360 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:14.798363 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:14.798365 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:14.798495 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:14.798859 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:34:14.798866 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:34:14.799089 (+ 223us) client_negotiation.cc:769] Sending connection context
0808 22:34:14.799249 (+ 160us) client_negotiation.cc:240] Negotiation successful
0808 22:34:14.799548 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":120,"threads_started":1}
I20260808 22:34:14.801434 22500 heartbeater.cc:347] Connected to a master server at 127.31.87.190:41657
I20260808 22:34:14.801641 22500 heartbeater.cc:464] Registering TS with master...
I20260808 22:34:14.802069 22500 heartbeater.cc:511] Master 127.31.87.190:41657 requested a full tablet report, sending...
I20260808 22:34:14.803059 21997 ts_manager.cc:194] Registered new tserver with Master: 0db1fc6a077a4abb8d1650a2b9540d75 (127.31.87.130:41469)
I20260808 22:34:14.803789 21997 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.130@KRBTEST.COM'} at 127.31.87.130:35447
W20260808 22:34:14.897908 22503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:34:14.898131 22503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:34:14.898171 22503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:34:14.902020 22503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:34:14.902071 22503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:34:14.902124 22503 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.131
I20260808 22:34:14.907111 22503 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:43005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab
--principal=kudu/127.31.87.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.131:44849
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.87.131
--webserver_port=42581
--tserver_master_addrs=127.31.87.190:41657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22503
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.131
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:34:14.908007 22503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:34:14.908908 22503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:34:14.917775 22510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:34:14.917786 22511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:34:14.917812 22516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:34:14.918136 22503 server_base.cc:1034] running on GCE node
I20260808 22:34:14.918495 22503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:34:14.919044 22503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:34:14.920200 22503 hybrid_clock.cc:648] HybridClock initialized: now 1786228454920168 us; error 42 us; skew 500 ppm
Aug 08 22:34:14 dist-test-slave-266d krb5kdc[21091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228454, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:34:14.922752 22503 init.cc:377] Logged in from keytab as kudu/127.31.87.131@KRBTEST.COM (short username kudu)
I20260808 22:34:14.923522 22503 webserver.cc:460] Webserver started at http://127.31.87.131:42581/ using document root <none> and password file <none>
I20260808 22:34:14.924002 22503 fs_manager.cc:359] Metadata directory not provided
I20260808 22:34:14.924069 22503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:34:14.927628 22503 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:34:14.929513 22528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:14.930621 22503 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:34:14.930734 22503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-2/wal
uuid: "d686840dc3584fbd9b3c7accdb6e26a3"
format_stamp: "Formatted at 2026-08-08 22:34:13 on dist-test-slave-266d"
server_key: "a7c67bcb19edb7a91b0d5b9b7bc6a0af"
server_key_iv: "481226f7429a5f096fd39bc0f0c6853b"
server_key_version: "encryptionkey@0"
I20260808 22:34:14.931219 22503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:34:14.964711 22503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:34:14.967586 22503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:34:14.967746 22503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:34:14.968290 22503 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:34:14.969120 22503 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:34:14.969169 22503 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:14.969223 22503 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:34:14.969251 22503 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:14.977286 22503 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.131:44849
I20260808 22:34:14.977339 22696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.131:44849 every 8 connection(s)
I20260808 22:34:14.978227 22503 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
Aug 08 22:34:14 dist-test-slave-266d krb5kdc[21091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228454, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.131@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:34:14.987959 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 22503
I20260808 22:34:14.990797 22289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:14.979835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:59799 (local address 127.31.87.190:41657)
0808 22:34:14.979946 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:14.979950 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:14.980760 (+ 810us) server_negotiation.cc:407] Connection header received
0808 22:34:14.982083 (+ 1323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:14.982087 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:14.982162 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:14.982232 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:34:14.983346 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:14.983853 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:14.984971 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:14.985236 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:14.985325 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:14.987440 (+ 2115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:34:14.987444 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:34:14.987447 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:34:14.987471 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:34:14.988914 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:14.989408 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:14.989412 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:14.989414 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:14.989494 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:14.989805 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:14.989808 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:14.989810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:14.989982 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:34:14.990067 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:14.990520 (+ 453us) server_negotiation.cc:299] Negotiation successful
0808 22:34:14.990636 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 22:34:14.991503 22699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:14.980151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.131:59799)
0808 22:34:14.980626 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:14.980666 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 22:34:14.981881 (+ 1215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:34:14.982414 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:34:14.982421 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:34:14.982770 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:34:14.983202 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:14.983215 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:14.983968 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:14.983971 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:14.984846 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:14.984857 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:14.985383 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:14.985387 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:14.985536 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:14.985983 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:34:14.985998 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:34:14.987275 (+ 1277us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:34:14.989016 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:14.989022 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:14.989035 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:14.989316 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:14.989575 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:14.989578 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:14.989581 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:14.989716 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:14.990079 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:34:14.990086 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:34:14.990302 (+ 216us) client_negotiation.cc:769] Sending connection context
0808 22:34:14.990459 (+ 157us) client_negotiation.cc:240] Negotiation successful
0808 22:34:14.990759 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":122,"threads_started":1}
I20260808 22:34:14.992652 22697 heartbeater.cc:347] Connected to a master server at 127.31.87.190:41657
I20260808 22:34:14.992872 22697 heartbeater.cc:464] Registering TS with master...
I20260808 22:34:14.993393 22697 heartbeater.cc:511] Master 127.31.87.190:41657 requested a full tablet report, sending...
I20260808 22:34:14.994524 21997 ts_manager.cc:194] Registered new tserver with Master: d686840dc3584fbd9b3c7accdb6e26a3 (127.31.87.131:44849)
I20260808 22:34:14.995177 21997 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.131@KRBTEST.COM'} at 127.31.87.131:59799
I20260808 22:34:14.996181 22702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:14.988609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38480 (local address 127.31.87.190:41657)
0808 22:34:14.988830 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:14.988833 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:14.989011 (+ 178us) server_negotiation.cc:407] Connection header received
0808 22:34:14.989171 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:14.989175 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:14.989216 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:14.989298 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:34:14.989991 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:14.990483 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:14.991394 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:14.991623 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:14.991708 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:14.993208 (+ 1500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:34:14.993212 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:34:14.993215 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:34:14.993240 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:34:14.994590 (+ 1350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:14.995023 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:14.995025 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:14.995026 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:14.995092 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:14.995355 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:14.995358 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:14.995360 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:14.995554 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:34:14.995724 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:14.995914 (+ 190us) server_negotiation.cc:299] Negotiation successful
0808 22:34:14.996030 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":74,"threads_started":1}
I20260808 22:34:14.997790 32094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:34:15.001657 22702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:14.998327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38486 (local address 127.31.87.190:41657)
0808 22:34:14.998432 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:14.998436 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:14.998581 (+ 145us) server_negotiation.cc:407] Connection header received
0808 22:34:14.998719 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:14.998722 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:14.998809 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:14.998907 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:34:14.999116 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:14.999587 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:15.000362 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:15.000584 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:15.000666 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:15.000958 (+ 292us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:34:15.001329 (+ 371us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:34:15.001433 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:15.001495 (+ 62us) server_negotiation.cc:299] Negotiation successful
0808 22:34:15.001546 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 22:34:15.002202 21997 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:38486:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:34:15.005318 21997 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:34:15.012789 22190 tablet_service.cc:1467] Processing CreateTablet for tablet 37cf708f0ea34e90b80755bdbe7d8489 (DEFAULT_TABLE table=test-table [id=d27be70d82d844a3a3a234b851c9f8e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:34:15.013479 22190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37cf708f0ea34e90b80755bdbe7d8489. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:34:15.016184 22293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:15.011494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.131:44849 (local address 127.0.0.1:56338)
0808 22:34:15.011602 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:15.011617 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:34:15.011694 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:34:15.012562 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:34:15.012564 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:34:15.012581 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:34:15.012671 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:15.012677 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:15.013976 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:15.013980 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:15.014965 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:15.014975 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:15.015734 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:15.015738 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:15.015836 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:15.015884 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 22:34:15.015973 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 22:34:15.016058 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":27}
I20260808 22:34:15.016335 22120 tablet_bootstrap.cc:492] T 37cf708f0ea34e90b80755bdbe7d8489 P 4355aa00a45b496793ff2602e299e94e: Bootstrap starting.
I20260808 22:34:15.016772 22711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:15.011692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56338 (local address 127.31.87.131:44849)
0808 22:34:15.012045 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:15.012057 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:15.012081 (+ 24us) server_negotiation.cc:407] Connection header received
0808 22:34:15.012161 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:15.012167 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:15.012380 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:15.012583 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:34:15.012768 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:15.013835 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:15.015081 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:15.015624 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:15.015730 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:15.015882 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:15.016003 (+ 121us) server_negotiation.cc:299] Negotiation successful
0808 22:34:15.016091 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":141,"threads_started":1}
I20260808 22:34:15.017235 22120 tablet_bootstrap.cc:654] T 37cf708f0ea34e90b80755bdbe7d8489 P 4355aa00a45b496793ff2602e299e94e: Neither blocks nor log segments found. Creating new log.
I20260808 22:34:15.017438 22712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:15.012055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.130:41469 (local address 127.0.0.1:58428)
0808 22:34:15.012318 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:15.012331 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:34:15.012417 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:34:15.013162 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:34:15.013165 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:34:15.013182 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:34:15.013254 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:15.013258 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:15.014429 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:15.014433 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:15.015527 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:15.015538 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:15.016957 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:15.016961 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:15.017206 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:15.017227 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 22:34:15.017289 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 22:34:15.017339 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":30,"thread_start_us":114,"threads_started":1}
I20260808 22:34:15.017625 22713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:15.012218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58428 (local address 127.31.87.130:41469)
0808 22:34:15.012620 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:15.012631 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:15.012652 (+ 21us) server_negotiation.cc:407] Connection header received
0808 22:34:15.012723 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:15.012728 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:15.012907 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:15.013036 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:34:15.013366 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:15.014283 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:15.015667 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:15.016694 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:15.016793 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:15.017197 (+ 404us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:15.017377 (+ 180us) server_negotiation.cc:299] Negotiation successful
0808 22:34:15.017474 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":98,"threads_started":1}
I20260808 22:34:15.019001 22120 tablet_bootstrap.cc:492] T 37cf708f0ea34e90b80755bdbe7d8489 P 4355aa00a45b496793ff2602e299e94e: No bootstrap required, opened a new log
I20260808 22:34:15.019093 22120 ts_tablet_manager.cc:1397] T 37cf708f0ea34e90b80755bdbe7d8489 P 4355aa00a45b496793ff2602e299e94e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260808 22:34:15.019520 22120 raft_consensus.cc:348] T 37cf708f0ea34e90b80755bdbe7d8489 P 4355aa00a45b496793ff2602e299e94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d686840dc3584fbd9b3c7accdb6e26a3" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 44849 } } peers { permanent_uuid: "0db1fc6a077a4abb8d1650a2b9540d75" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41469 } } peers { permanent_uuid: "4355aa00a45b496793ff2602e299e94e" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 38117 } }
I20260808 22:34:15.019636 22120 raft_consensus.cc:374] T 37cf708f0ea34e90b80755bdbe7d8489 P 4355aa00a45b496793ff2602e299e94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:34:15.019675 22120 raft_consensus.cc:729] T 37cf708f0ea34e90b80755bdbe7d8489 P 4355aa00a45b496793ff2602e299e94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4355aa00a45b496793ff2602e299e94e, State: Initialized, Role: FOLLOWER
I20260808 22:34:15.019812 22120 consensus_queue.cc:260] T 37cf708f0ea34e90b80755bdbe7d8489 P 4355aa00a45b496793ff2602e299e94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d686840dc3584fbd9b3c7accdb6e26a3" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 44849 } } peers { permanent_uuid: "0db1fc6a077a4abb8d1650a2b9540d75" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41469 } } peers { permanent_uuid: "4355aa00a45b496793ff2602e299e94e" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 38117 } }
I20260808 22:34:15.020180 22120 ts_tablet_manager.cc:1428] T 37cf708f0ea34e90b80755bdbe7d8489 P 4355aa00a45b496793ff2602e299e94e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 22:34:15.020090 22602 tablet_service.cc:1467] Processing CreateTablet for tablet 37cf708f0ea34e90b80755bdbe7d8489 (DEFAULT_TABLE table=test-table [id=d27be70d82d844a3a3a234b851c9f8e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:34:15.020556 22397 tablet_service.cc:1467] Processing CreateTablet for tablet 37cf708f0ea34e90b80755bdbe7d8489 (DEFAULT_TABLE table=test-table [id=d27be70d82d844a3a3a234b851c9f8e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:34:15.021134 22602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37cf708f0ea34e90b80755bdbe7d8489. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:34:15.021577 22397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37cf708f0ea34e90b80755bdbe7d8489. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:34:15.027359 22715 tablet_bootstrap.cc:492] T 37cf708f0ea34e90b80755bdbe7d8489 P d686840dc3584fbd9b3c7accdb6e26a3: Bootstrap starting.
I20260808 22:34:15.027421 22716 tablet_bootstrap.cc:492] T 37cf708f0ea34e90b80755bdbe7d8489 P 0db1fc6a077a4abb8d1650a2b9540d75: Bootstrap starting.
I20260808 22:34:15.029785 22716 tablet_bootstrap.cc:654] T 37cf708f0ea34e90b80755bdbe7d8489 P 0db1fc6a077a4abb8d1650a2b9540d75: Neither blocks nor log segments found. Creating new log.
I20260808 22:34:15.029896 22715 tablet_bootstrap.cc:654] T 37cf708f0ea34e90b80755bdbe7d8489 P d686840dc3584fbd9b3c7accdb6e26a3: Neither blocks nor log segments found. Creating new log.
I20260808 22:34:15.030489 22716 log.cc:824] T 37cf708f0ea34e90b80755bdbe7d8489 P 0db1fc6a077a4abb8d1650a2b9540d75: Log is configured to *not* fsync() on all Append() calls
I20260808 22:34:15.030506 22715 log.cc:824] T 37cf708f0ea34e90b80755bdbe7d8489 P d686840dc3584fbd9b3c7accdb6e26a3: Log is configured to *not* fsync() on all Append() calls
I20260808 22:34:15.032212 22715 tablet_bootstrap.cc:492] T 37cf708f0ea34e90b80755bdbe7d8489 P d686840dc3584fbd9b3c7accdb6e26a3: No bootstrap required, opened a new log
I20260808 22:34:15.032212 22716 tablet_bootstrap.cc:492] T 37cf708f0ea34e90b80755bdbe7d8489 P 0db1fc6a077a4abb8d1650a2b9540d75: No bootstrap required, opened a new log
I20260808 22:34:15.032372 22715 ts_tablet_manager.cc:1397] T 37cf708f0ea34e90b80755bdbe7d8489 P d686840dc3584fbd9b3c7accdb6e26a3: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 22:34:15.032375 22716 ts_tablet_manager.cc:1397] T 37cf708f0ea34e90b80755bdbe7d8489 P 0db1fc6a077a4abb8d1650a2b9540d75: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 22:34:15.035167 22715 raft_consensus.cc:348] T 37cf708f0ea34e90b80755bdbe7d8489 P d686840dc3584fbd9b3c7accdb6e26a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d686840dc3584fbd9b3c7accdb6e26a3" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 44849 } } peers { permanent_uuid: "0db1fc6a077a4abb8d1650a2b9540d75" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41469 } } peers { permanent_uuid: "4355aa00a45b496793ff2602e299e94e" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 38117 } }
I20260808 22:34:15.035197 22716 raft_consensus.cc:348] T 37cf708f0ea34e90b80755bdbe7d8489 P 0db1fc6a077a4abb8d1650a2b9540d75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d686840dc3584fbd9b3c7accdb6e26a3" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 44849 } } peers { permanent_uuid: "0db1fc6a077a4abb8d1650a2b9540d75" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41469 } } peers { permanent_uuid: "4355aa00a45b496793ff2602e299e94e" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 38117 } }
I20260808 22:34:15.035411 22716 raft_consensus.cc:374] T 37cf708f0ea34e90b80755bdbe7d8489 P 0db1fc6a077a4abb8d1650a2b9540d75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:34:15.035413 22715 raft_consensus.cc:374] T 37cf708f0ea34e90b80755bdbe7d8489 P d686840dc3584fbd9b3c7accdb6e26a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:34:15.035465 22716 raft_consensus.cc:729] T 37cf708f0ea34e90b80755bdbe7d8489 P 0db1fc6a077a4abb8d1650a2b9540d75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0db1fc6a077a4abb8d1650a2b9540d75, State: Initialized, Role: FOLLOWER
I20260808 22:34:15.035467 22715 raft_consensus.cc:729] T 37cf708f0ea34e90b80755bdbe7d8489 P d686840dc3584fbd9b3c7accdb6e26a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d686840dc3584fbd9b3c7accdb6e26a3, State: Initialized, Role: FOLLOWER
I20260808 22:34:15.035881 22715 consensus_queue.cc:260] T 37cf708f0ea34e90b80755bdbe7d8489 P d686840dc3584fbd9b3c7accdb6e26a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d686840dc3584fbd9b3c7accdb6e26a3" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 44849 } } peers { permanent_uuid: "0db1fc6a077a4abb8d1650a2b9540d75" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41469 } } peers { permanent_uuid: "4355aa00a45b496793ff2602e299e94e" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 38117 } }
I20260808 22:34:15.035881 22716 consensus_queue.cc:260] T 37cf708f0ea34e90b80755bdbe7d8489 P 0db1fc6a077a4abb8d1650a2b9540d75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d686840dc3584fbd9b3c7accdb6e26a3" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 44849 } } peers { permanent_uuid: "0db1fc6a077a4abb8d1650a2b9540d75" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41469 } } peers { permanent_uuid: "4355aa00a45b496793ff2602e299e94e" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 38117 } }
I20260808 22:34:15.036510 22697 heartbeater.cc:503] Master 127.31.87.190:41657 was elected leader, sending a full tablet report...
I20260808 22:34:15.036732 22715 ts_tablet_manager.cc:1428] T 37cf708f0ea34e90b80755bdbe7d8489 P d686840dc3584fbd9b3c7accdb6e26a3: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260808 22:34:15.036760 22716 ts_tablet_manager.cc:1428] T 37cf708f0ea34e90b80755bdbe7d8489 P 0db1fc6a077a4abb8d1650a2b9540d75: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260808 22:34:15.037078 22500 heartbeater.cc:503] Master 127.31.87.190:41657 was elected leader, sending a full tablet report...
W20260808 22:34:15.037837 22501 tablet.cc:2369] T 37cf708f0ea34e90b80755bdbe7d8489 P 0db1fc6a077a4abb8d1650a2b9540d75: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 22:34:15.110718 22287 tablet.cc:2369] T 37cf708f0ea34e90b80755bdbe7d8489 P 4355aa00a45b496793ff2602e299e94e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:34:15.119428 22720 raft_consensus.cc:482] T 37cf708f0ea34e90b80755bdbe7d8489 P 0db1fc6a077a4abb8d1650a2b9540d75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:34:15.119607 22720 raft_consensus.cc:504] T 37cf708f0ea34e90b80755bdbe7d8489 P 0db1fc6a077a4abb8d1650a2b9540d75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d686840dc3584fbd9b3c7accdb6e26a3" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 44849 } } peers { permanent_uuid: "0db1fc6a077a4abb8d1650a2b9540d75" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41469 } } peers { permanent_uuid: "4355aa00a45b496793ff2602e299e94e" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 38117 } }
I20260808 22:34:15.120666 22720 leader_election.cc:288] T 37cf708f0ea34e90b80755bdbe7d8489 P 0db1fc6a077a4abb8d1650a2b9540d75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d686840dc3584fbd9b3c7accdb6e26a3 (127.31.87.131:44849), 4355aa00a45b496793ff2602e299e94e (127.31.87.129:38117)
I20260808 22:34:15.124641 22502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:15.120878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.131:44849 (local address 127.31.87.130:47809)
0808 22:34:15.121003 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:15.121018 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:34:15.121130 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:34:15.121384 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:34:15.121387 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:34:15.121407 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:34:15.121531 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:15.121539 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:15.122464 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:15.122466 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:15.123570 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:15.123581 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:15.124314 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:15.124317 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:15.124395 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:15.124415 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 22:34:15.124470 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 22:34:15.124522 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":1}
I20260808 22:34:15.124686 22711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:15.120974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:47809 (local address 127.31.87.131:44849)
0808 22:34:15.121110 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:15.121115 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:15.121131 (+ 16us) server_negotiation.cc:407] Connection header received
0808 22:34:15.121225 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:15.121228 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:15.121280 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:15.121360 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:34:15.121645 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:15.122362 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:15.123703 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:15.124210 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:15.124323 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:15.124376 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:15.124499 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 22:34:15.124563 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 22:34:15.125113 22296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:15.121316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:45683 (local address 127.31.87.129:38117)
0808 22:34:15.121436 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:15.121440 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:15.121651 (+ 211us) server_negotiation.cc:407] Connection header received
0808 22:34:15.121772 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:15.121776 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:15.121836 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:15.121924 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:34:15.122119 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:15.122727 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:15.123977 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:15.124576 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:15.124683 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:15.124820 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:15.124911 (+ 91us) server_negotiation.cc:299] Negotiation successful
0808 22:34:15.124978 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 22:34:15.125133 22724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:15.121200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:38117 (local address 127.31.87.130:45683)
0808 22:34:15.121573 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:15.121584 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:34:15.121671 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:34:15.121938 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:34:15.121940 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:34:15.121951 (+ 11us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:34:15.122040 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:15.122044 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:15.122831 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:15.122835 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:15.123876 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:15.123884 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:15.124707 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:15.124711 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:15.124800 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:15.124826 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 22:34:15.124898 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 22:34:15.124967 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":104,"threads_started":1}
I20260808 22:34:15.125298 22630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37cf708f0ea34e90b80755bdbe7d8489" candidate_uuid: "0db1fc6a077a4abb8d1650a2b9540d75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d686840dc3584fbd9b3c7accdb6e26a3" is_pre_election: true
I20260808 22:34:15.125591 22630 raft_consensus.cc:2454] T 37cf708f0ea34e90b80755bdbe7d8489 P d686840dc3584fbd9b3c7accdb6e26a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0db1fc6a077a4abb8d1650a2b9540d75 in term 0.
I20260808 22:34:15.125589 22215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37cf708f0ea34e90b80755bdbe7d8489" candidate_uuid: "0db1fc6a077a4abb8d1650a2b9540d75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4355aa00a45b496793ff2602e299e94e" is_pre_election: true
I20260808 22:34:15.125833 22215 raft_consensus.cc:2454] T 37cf708f0ea34e90b80755bdbe7d8489 P 4355aa00a45b496793ff2602e299e94e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0db1fc6a077a4abb8d1650a2b9540d75 in term 0.
I20260808 22:34:15.126065 22322 leader_election.cc:302] T 37cf708f0ea34e90b80755bdbe7d8489 P 0db1fc6a077a4abb8d1650a2b9540d75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0db1fc6a077a4abb8d1650a2b9540d75, d686840dc3584fbd9b3c7accdb6e26a3; no voters:
I20260808 22:34:15.126350 22720 raft_consensus.cc:2781] T 37cf708f0ea34e90b80755bdbe7d8489 P 0db1fc6a077a4abb8d1650a2b9540d75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:34:15.126434 22720 raft_consensus.cc:482] T 37cf708f0ea34e90b80755bdbe7d8489 P 0db1fc6a077a4abb8d1650a2b9540d75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:34:15.126482 22720 raft_consensus.cc:3037] T 37cf708f0ea34e90b80755bdbe7d8489 P 0db1fc6a077a4abb8d1650a2b9540d75 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:34:15.127524 22720 raft_consensus.cc:504] T 37cf708f0ea34e90b80755bdbe7d8489 P 0db1fc6a077a4abb8d1650a2b9540d75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d686840dc3584fbd9b3c7accdb6e26a3" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 44849 } } peers { permanent_uuid: "0db1fc6a077a4abb8d1650a2b9540d75" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41469 } } peers { permanent_uuid: "4355aa00a45b496793ff2602e299e94e" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 38117 } }
I20260808 22:34:15.127821 22720 leader_election.cc:288] T 37cf708f0ea34e90b80755bdbe7d8489 P 0db1fc6a077a4abb8d1650a2b9540d75 [CANDIDATE]: Term 1 election: Requested vote from peers d686840dc3584fbd9b3c7accdb6e26a3 (127.31.87.131:44849), 4355aa00a45b496793ff2602e299e94e (127.31.87.129:38117)
I20260808 22:34:15.128124 22630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37cf708f0ea34e90b80755bdbe7d8489" candidate_uuid: "0db1fc6a077a4abb8d1650a2b9540d75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d686840dc3584fbd9b3c7accdb6e26a3"
I20260808 22:34:15.128177 22215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37cf708f0ea34e90b80755bdbe7d8489" candidate_uuid: "0db1fc6a077a4abb8d1650a2b9540d75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4355aa00a45b496793ff2602e299e94e"
I20260808 22:34:15.128247 22630 raft_consensus.cc:3037] T 37cf708f0ea34e90b80755bdbe7d8489 P d686840dc3584fbd9b3c7accdb6e26a3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:34:15.128274 22215 raft_consensus.cc:3037] T 37cf708f0ea34e90b80755bdbe7d8489 P 4355aa00a45b496793ff2602e299e94e [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:34:15.129493 22630 raft_consensus.cc:2454] T 37cf708f0ea34e90b80755bdbe7d8489 P d686840dc3584fbd9b3c7accdb6e26a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0db1fc6a077a4abb8d1650a2b9540d75 in term 1.
I20260808 22:34:15.129493 22215 raft_consensus.cc:2454] T 37cf708f0ea34e90b80755bdbe7d8489 P 4355aa00a45b496793ff2602e299e94e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0db1fc6a077a4abb8d1650a2b9540d75 in term 1.
I20260808 22:34:15.129755 22322 leader_election.cc:302] T 37cf708f0ea34e90b80755bdbe7d8489 P 0db1fc6a077a4abb8d1650a2b9540d75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0db1fc6a077a4abb8d1650a2b9540d75, d686840dc3584fbd9b3c7accdb6e26a3; no voters:
I20260808 22:34:15.129901 22720 raft_consensus.cc:2781] T 37cf708f0ea34e90b80755bdbe7d8489 P 0db1fc6a077a4abb8d1650a2b9540d75 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:34:15.130097 22720 raft_consensus.cc:686] T 37cf708f0ea34e90b80755bdbe7d8489 P 0db1fc6a077a4abb8d1650a2b9540d75 [term 1 LEADER]: Becoming Leader. State: Replica: 0db1fc6a077a4abb8d1650a2b9540d75, State: Running, Role: LEADER
I20260808 22:34:15.130414 22720 consensus_queue.cc:237] T 37cf708f0ea34e90b80755bdbe7d8489 P 0db1fc6a077a4abb8d1650a2b9540d75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d686840dc3584fbd9b3c7accdb6e26a3" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 44849 } } peers { permanent_uuid: "0db1fc6a077a4abb8d1650a2b9540d75" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41469 } } peers { permanent_uuid: "4355aa00a45b496793ff2602e299e94e" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 38117 } }
I20260808 22:34:15.133257 21996 catalog_manager.cc:5495] T 37cf708f0ea34e90b80755bdbe7d8489 P 0db1fc6a077a4abb8d1650a2b9540d75 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0db1fc6a077a4abb8d1650a2b9540d75 (127.31.87.130). New cstate: current_term: 1 leader_uuid: "0db1fc6a077a4abb8d1650a2b9540d75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d686840dc3584fbd9b3c7accdb6e26a3" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 44849 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0db1fc6a077a4abb8d1650a2b9540d75" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4355aa00a45b496793ff2602e299e94e" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 38117 } health_report { overall_health: UNKNOWN } } }
I20260808 22:34:15.147962 22713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:15.144923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58436 (local address 127.31.87.130:41469)
0808 22:34:15.145036 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:15.145040 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:15.145053 (+ 13us) server_negotiation.cc:407] Connection header received
0808 22:34:15.145197 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:15.145199 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:15.145246 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:15.145325 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:34:15.145530 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:15.146136 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:15.146848 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:15.147041 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:15.147108 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:15.147309 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:34:15.147677 (+ 368us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:34:15.147758 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:15.147804 (+ 46us) server_negotiation.cc:299] Negotiation successful
0808 22:34:15.147849 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 22:34:15.153146 22215 raft_consensus.cc:1261] T 37cf708f0ea34e90b80755bdbe7d8489 P 4355aa00a45b496793ff2602e299e94e [term 1 FOLLOWER]: Refusing update from remote peer 0db1fc6a077a4abb8d1650a2b9540d75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:34:15.153164 22630 raft_consensus.cc:1261] T 37cf708f0ea34e90b80755bdbe7d8489 P d686840dc3584fbd9b3c7accdb6e26a3 [term 1 FOLLOWER]: Refusing update from remote peer 0db1fc6a077a4abb8d1650a2b9540d75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:34:15.153760 22720 consensus_queue.cc:1035] T 37cf708f0ea34e90b80755bdbe7d8489 P 0db1fc6a077a4abb8d1650a2b9540d75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4355aa00a45b496793ff2602e299e94e" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 38117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:34:15.153936 22728 consensus_queue.cc:1035] T 37cf708f0ea34e90b80755bdbe7d8489 P 0db1fc6a077a4abb8d1650a2b9540d75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d686840dc3584fbd9b3c7accdb6e26a3" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 44849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:34:15.160929 22732 mvcc.cc:205] Tried to move back new op lower bound from 7316391752301195264 to 7316391752216436736. Current Snapshot: MvccSnapshot[applied={T|T < 7316391752301195264}]
I20260808 22:34:15.167155 21996 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:38486:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:34:15.167305 21996 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:38486:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:34:15.169481 21996 catalog_manager.cc:5782] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439: Sending DeleteTablet for 3 replicas of tablet 37cf708f0ea34e90b80755bdbe7d8489
I20260808 22:34:15.170130 22397 tablet_service.cc:1514] Processing DeleteTablet for tablet 37cf708f0ea34e90b80755bdbe7d8489 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:34:15 UTC) from {username='kudu', principal='kudu/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:58428
I20260808 22:34:15.170135 22602 tablet_service.cc:1514] Processing DeleteTablet for tablet 37cf708f0ea34e90b80755bdbe7d8489 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:34:15 UTC) from {username='kudu', principal='kudu/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:56338
I20260808 22:34:15.170207 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 22056
I20260808 22:34:15.170539 22741 tablet_replica.cc:330] T 37cf708f0ea34e90b80755bdbe7d8489 P 0db1fc6a077a4abb8d1650a2b9540d75: stopping tablet replica
I20260808 22:34:15.170794 22741 raft_consensus.cc:2229] T 37cf708f0ea34e90b80755bdbe7d8489 P 0db1fc6a077a4abb8d1650a2b9540d75 [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:34:15.171062 22741 raft_consensus.cc:2258] T 37cf708f0ea34e90b80755bdbe7d8489 P 0db1fc6a077a4abb8d1650a2b9540d75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:34:15.181722 22741 ts_tablet_manager.cc:1905] T 37cf708f0ea34e90b80755bdbe7d8489 P 0db1fc6a077a4abb8d1650a2b9540d75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:34:15.183825 22741 ts_tablet_manager.cc:1918] T 37cf708f0ea34e90b80755bdbe7d8489 P 0db1fc6a077a4abb8d1650a2b9540d75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:34:15.183911 22741 log.cc:1196] T 37cf708f0ea34e90b80755bdbe7d8489 P 0db1fc6a077a4abb8d1650a2b9540d75: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-1/wal/wals/37cf708f0ea34e90b80755bdbe7d8489
I20260808 22:34:15.184204 22741 ts_tablet_manager.cc:1939] T 37cf708f0ea34e90b80755bdbe7d8489 P 0db1fc6a077a4abb8d1650a2b9540d75: Deleting consensus metadata
I20260808 22:34:15.185190 21975 catalog_manager.cc:4841] TS 0db1fc6a077a4abb8d1650a2b9540d75 (127.31.87.130:41469): tablet 37cf708f0ea34e90b80755bdbe7d8489 (table test-table [id=d27be70d82d844a3a3a234b851c9f8e0]) successfully deleted
I20260808 22:34:15.185326 22744 tablet_replica.cc:330] T 37cf708f0ea34e90b80755bdbe7d8489 P d686840dc3584fbd9b3c7accdb6e26a3: stopping tablet replica
I20260808 22:34:15.185601 22744 raft_consensus.cc:2229] T 37cf708f0ea34e90b80755bdbe7d8489 P d686840dc3584fbd9b3c7accdb6e26a3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:34:15.185887 22744 raft_consensus.cc:2258] T 37cf708f0ea34e90b80755bdbe7d8489 P d686840dc3584fbd9b3c7accdb6e26a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:34:15.187022 22744 ts_tablet_manager.cc:1905] T 37cf708f0ea34e90b80755bdbe7d8489 P d686840dc3584fbd9b3c7accdb6e26a3: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 22:34:15.188117 21979 connection.cc:535] client connection to 127.31.87.129:38117 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 22:34:15.188258 21979 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 22:34:15.189190 22744 ts_tablet_manager.cc:1918] T 37cf708f0ea34e90b80755bdbe7d8489 P d686840dc3584fbd9b3c7accdb6e26a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:34:15.189294 22744 log.cc:1196] T 37cf708f0ea34e90b80755bdbe7d8489 P d686840dc3584fbd9b3c7accdb6e26a3: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-2/wal/wals/37cf708f0ea34e90b80755bdbe7d8489
W20260808 22:34:15.189399 22712 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:34:15.189081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:38117 (local address 127.0.0.1:45744)
0808 22:34:15.189224 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:15.189312 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.129:38117: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 22:34:15.189633 22744 ts_tablet_manager.cc:1939] T 37cf708f0ea34e90b80755bdbe7d8489 P d686840dc3584fbd9b3c7accdb6e26a3: Deleting consensus metadata
W20260808 22:34:15.189523 21979 catalog_manager.cc:4584] TS 4355aa00a45b496793ff2602e299e94e (127.31.87.129:38117): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 37cf708f0ea34e90b80755bdbe7d8489: Network error: Client connection negotiation failed: client connection to 127.31.87.129:38117: connect: Connection reset by peer (error 104)
I20260808 22:34:15.190369 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 22290
I20260808 22:34:15.192272 21979 catalog_manager.cc:4841] TS d686840dc3584fbd9b3c7accdb6e26a3 (127.31.87.131:44849): tablet 37cf708f0ea34e90b80755bdbe7d8489 (table test-table [id=d27be70d82d844a3a3a234b851c9f8e0]) successfully deleted
I20260808 22:34:15.199810 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 22503
I20260808 22:34:15.223492 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 21947
I20260808 22:34:15.236382 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:41657
--webserver_interface=127.31.87.190
--webserver_port=37393
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:43005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:41657
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:34:15.343245 22746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:34:15.343463 22746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:34:15.343500 22746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:34:15.347317 22746 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:34:15.347364 22746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:34:15.347381 22746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:34:15.347401 22746 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:34:15.347415 22746 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:34:15.353252 22746 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:43005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:41657
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:41657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.87.190
--webserver_port=37393
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22746
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:34:15.354120 22746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:34:15.355008 22746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:34:15.362942 22752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:34:15.363032 22755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:34:15.362944 22751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:34:15.363296 22746 server_base.cc:1034] running on GCE node
I20260808 22:34:15.363847 22746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:34:15.364884 22746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:34:15.366045 22746 hybrid_clock.cc:648] HybridClock initialized: now 1786228455366017 us; error 45 us; skew 500 ppm
Aug 08 22:34:15 dist-test-slave-266d krb5kdc[21091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228455, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:34:15.373831 22746 init.cc:377] Logged in from keytab as kudu/127.31.87.190@KRBTEST.COM (short username kudu)
I20260808 22:34:15.374702 22746 webserver.cc:460] Webserver started at http://127.31.87.190:37393/ using document root <none> and password file <none>
I20260808 22:34:15.375219 22746 fs_manager.cc:359] Metadata directory not provided
I20260808 22:34:15.375291 22746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:34:15.379024 22746 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 22:34:15.380991 22762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:15.382187 22746 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:34:15.382299 22746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "aa7651551d8d498890f0984c21549439"
format_stamp: "Formatted at 2026-08-08 22:34:12 on dist-test-slave-266d"
server_key: "8c08cb5db44f911c46876007b3896cb8"
server_key_iv: "4ab580b9b3a9561b384123b0f8a3a859"
server_key_version: "encryptionkey@0"
I20260808 22:34:15.382789 22746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:34:15.404647 22746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:34:15.407235 22746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:34:15.407415 22746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:34:15.414582 22746 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.190:41657
I20260808 22:34:15.414811 22840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.190:41657 every 8 connection(s)
I20260808 22:34:15.415678 22746 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:34:15.419659 22841 sys_catalog.cc:263] Verifying existing consensus state
I20260808 22:34:15.421257 22841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439: Bootstrap starting.
I20260808 22:34:15.422241 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 22746
I20260808 22:34:15.422749 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.129:38117
--local_ip_for_outbound_sockets=127.31.87.129
--tserver_master_addrs=127.31.87.190:41657
--webserver_port=34235
--webserver_interface=127.31.87.129
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab
--principal=kudu/127.31.87.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:43005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 22:34:15.438477 22841 log.cc:824] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439: Log is configured to *not* fsync() on all Append() calls
I20260808 22:34:15.448997 22841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 22:34:15.449309 22841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439: Bootstrap complete.
I20260808 22:34:15.452906 22841 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa7651551d8d498890f0984c21549439" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41657 } }
I20260808 22:34:15.453291 22841 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa7651551d8d498890f0984c21549439, State: Initialized, Role: FOLLOWER
I20260808 22:34:15.453752 22841 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa7651551d8d498890f0984c21549439" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41657 } }
I20260808 22:34:15.453864 22841 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:34:15.453925 22841 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:34:15.454011 22841 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439 [term 2 FOLLOWER]: Advancing to term 3
I20260808 22:34:15.455817 22841 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa7651551d8d498890f0984c21549439" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41657 } }
I20260808 22:34:15.456101 22841 leader_election.cc:302] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa7651551d8d498890f0984c21549439; no voters:
I20260808 22:34:15.456377 22841 leader_election.cc:288] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 22:34:15.456457 22847 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 22:34:15.456667 22847 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439 [term 3 LEADER]: Becoming Leader. State: Replica: aa7651551d8d498890f0984c21549439, State: Running, Role: LEADER
I20260808 22:34:15.456951 22847 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa7651551d8d498890f0984c21549439" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41657 } }
I20260808 22:34:15.457515 22841 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:34:15.457876 22847 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa7651551d8d498890f0984c21549439. Latest consensus state: current_term: 3 leader_uuid: "aa7651551d8d498890f0984c21549439" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa7651551d8d498890f0984c21549439" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41657 } } }
I20260808 22:34:15.457981 22847 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439 [sys.catalog]: This master's current role is: LEADER
I20260808 22:34:15.458374 22851 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "aa7651551d8d498890f0984c21549439" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa7651551d8d498890f0984c21549439" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 41657 } } }
I20260808 22:34:15.458576 22851 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa7651551d8d498890f0984c21549439 [sys.catalog]: This master's current role is: LEADER
I20260808 22:34:15.459658 22854 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:34:15.463459 22854 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:34:15.463970 22854 catalog_manager.cc:1259] Loaded cluster ID: 096d94a0830e4ccc8476217ffb722cc7
I20260808 22:34:15.464020 22854 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260808 22:34:15.464838 22854 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f197b8659ad google::LogMessage::Fail() at ??:0
@ 0x7f197b86888a google::LogMessage::SendToLog() at ??:0
@ 0x7f197b8654de google::LogMessage::Flush() at ??:0
@ 0x7f197b866a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f19861a4fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f19861a5d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f19861a2579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f19861e6640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f197dab73d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f197dab7c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f197dab93f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f197daa8512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f197f0ec6db start_thread at ??:0
@ 0x7f197a36361f clone at ??:0
W20260808 22:34:15.570950 22844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:34:15.571305 22844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:34:15.571381 22844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:34:15.577186 22844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:34:15.577263 22844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:34:15.577360 22844 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.129
I20260808 22:34:15.584725 22844 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:43005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab
--principal=kudu/127.31.87.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.129:38117
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.87.129
--webserver_port=34235
--tserver_master_addrs=127.31.87.190:41657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22844
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.129
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:34:15.586037 22844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:34:15.587380 22844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:34:15.599694 22895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:34:15.599833 22844 server_base.cc:1034] running on GCE node
W20260808 22:34:15.599725 22897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:34:15.599701 22899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:34:15.600469 22844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:34:15.601078 22844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:34:15.602283 22844 hybrid_clock.cc:648] HybridClock initialized: now 1786228455602254 us; error 35 us; skew 500 ppm
Aug 08 22:34:15 dist-test-slave-266d krb5kdc[21091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228455, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:34:15.604961 22844 init.cc:377] Logged in from keytab as kudu/127.31.87.129@KRBTEST.COM (short username kudu)
I20260808 22:34:15.605819 22844 webserver.cc:460] Webserver started at http://127.31.87.129:34235/ using document root <none> and password file <none>
I20260808 22:34:15.606343 22844 fs_manager.cc:359] Metadata directory not provided
I20260808 22:34:15.606408 22844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:34:15.610143 22844 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260808 22:34:15.612272 22908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:15.613583 22844 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.002s
I20260808 22:34:15.613694 22844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-0/wal
uuid: "4355aa00a45b496793ff2602e299e94e"
format_stamp: "Formatted at 2026-08-08 22:34:13 on dist-test-slave-266d"
server_key: "fcb44dde4838b861fb64d9a0e7ef8919"
server_key_iv: "8a0b22cc07e34d0e72cb43451eaa5325"
server_key_version: "encryptionkey@0"
I20260808 22:34:15.614189 22844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:34:15.627063 22844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:34:15.629912 22844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:34:15.630067 22844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:34:15.630574 22844 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:34:15.631600 22916 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 22:34:15.636731 22844 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 22:34:15.636790 22844 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260808 22:34:15.636844 22844 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 22:34:15.638366 22844 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 22:34:15.638408 22844 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260808 22:34:15.638746 22916 tablet_bootstrap.cc:492] T 37cf708f0ea34e90b80755bdbe7d8489 P 4355aa00a45b496793ff2602e299e94e: Bootstrap starting.
I20260808 22:34:15.647861 22844 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.129:38117
I20260808 22:34:15.648087 23073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.129:38117 every 8 connection(s)
I20260808 22:34:15.648958 22844 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
I20260808 22:34:15.650197 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 22844
I20260808 22:34:15.650590 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.130:41469
--local_ip_for_outbound_sockets=127.31.87.130
--tserver_master_addrs=127.31.87.190:41657
--webserver_port=44951
--webserver_interface=127.31.87.130
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab
--principal=kudu/127.31.87.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:43005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 22:34:15.661944 22916 log.cc:824] T 37cf708f0ea34e90b80755bdbe7d8489 P 4355aa00a45b496793ff2602e299e94e: Log is configured to *not* fsync() on all Append() calls
I20260808 22:34:15.670717 22916 tablet_bootstrap.cc:492] T 37cf708f0ea34e90b80755bdbe7d8489 P 4355aa00a45b496793ff2602e299e94e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 22:34:15.671113 22916 tablet_bootstrap.cc:492] T 37cf708f0ea34e90b80755bdbe7d8489 P 4355aa00a45b496793ff2602e299e94e: Bootstrap complete.
I20260808 22:34:15.672426 22916 ts_tablet_manager.cc:1397] T 37cf708f0ea34e90b80755bdbe7d8489 P 4355aa00a45b496793ff2602e299e94e: Time spent bootstrapping tablet: real 0.034s user 0.024s sys 0.004s
I20260808 22:34:15.675737 22916 raft_consensus.cc:348] T 37cf708f0ea34e90b80755bdbe7d8489 P 4355aa00a45b496793ff2602e299e94e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d686840dc3584fbd9b3c7accdb6e26a3" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 44849 } } peers { permanent_uuid: "0db1fc6a077a4abb8d1650a2b9540d75" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41469 } } peers { permanent_uuid: "4355aa00a45b496793ff2602e299e94e" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 38117 } }
I20260808 22:34:15.676281 22916 raft_consensus.cc:729] T 37cf708f0ea34e90b80755bdbe7d8489 P 4355aa00a45b496793ff2602e299e94e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4355aa00a45b496793ff2602e299e94e, State: Initialized, Role: FOLLOWER
I20260808 22:34:15.676816 22916 consensus_queue.cc:260] T 37cf708f0ea34e90b80755bdbe7d8489 P 4355aa00a45b496793ff2602e299e94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d686840dc3584fbd9b3c7accdb6e26a3" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 44849 } } peers { permanent_uuid: "0db1fc6a077a4abb8d1650a2b9540d75" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41469 } } peers { permanent_uuid: "4355aa00a45b496793ff2602e299e94e" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 38117 } }
I20260808 22:34:15.677898 22916 ts_tablet_manager.cc:1428] T 37cf708f0ea34e90b80755bdbe7d8489 P 4355aa00a45b496793ff2602e299e94e: Time spent starting tablet: real 0.005s user 0.004s sys 0.004s
W20260808 22:34:15.696098 23079 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:34:15.650935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.129:54027)
0808 22:34:15.651479 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:15.651533 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 22:34:15.652476 (+ 943us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:34:15.695348 (+ 42872us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41657: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":148,"threads_started":1}
W20260808 22:34:15.696667 23074 heartbeater.cc:644] Failed to heartbeat to 127.31.87.190:41657 (0 consecutive failures): Network error: Failed to ping master at 127.31.87.190:41657: Client connection negotiation failed: client connection to 127.31.87.190:41657: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 22:34:15.697398 23079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:15.697159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.129:33917)
0808 22:34:15.697281 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:15.697344 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 22:34:15.698045 23079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:15.697854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.129:57801)
0808 22:34:15.697955 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:15.697998 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 22:34:15.698196 23074 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 22:34:15.763074 23081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:34:15.763294 23081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:34:15.763337 23081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:34:15.767350 23081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:34:15.767400 23081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:34:15.767455 23081 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.130
I20260808 22:34:15.772373 23081 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:43005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab
--principal=kudu/127.31.87.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.130:41469
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.87.130
--webserver_port=44951
--tserver_master_addrs=127.31.87.190:41657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23081
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.130
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:34:15.773295 23081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:34:15.774195 23081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:34:15.783162 23095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:34:15.783212 23093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:34:15.783245 23092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:34:15.783305 23081 server_base.cc:1034] running on GCE node
I20260808 22:34:15.783771 23081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:34:15.784353 23081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:34:15.785504 23081 hybrid_clock.cc:648] HybridClock initialized: now 1786228455785483 us; error 36 us; skew 500 ppm
Aug 08 22:34:15 dist-test-slave-266d krb5kdc[21091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228455, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:34:15.787977 23081 init.cc:377] Logged in from keytab as kudu/127.31.87.130@KRBTEST.COM (short username kudu)
I20260808 22:34:15.788806 23081 webserver.cc:460] Webserver started at http://127.31.87.130:44951/ using document root <none> and password file <none>
I20260808 22:34:15.789366 23081 fs_manager.cc:359] Metadata directory not provided
I20260808 22:34:15.789434 23081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:34:15.793126 23081 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 22:34:15.795073 23105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:15.796223 23081 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:34:15.796332 23081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-1/wal
uuid: "0db1fc6a077a4abb8d1650a2b9540d75"
format_stamp: "Formatted at 2026-08-08 22:34:13 on dist-test-slave-266d"
server_key: "e06bdce1b696c3ead6da19debab55c73"
server_key_iv: "5cf8f2a82dbecb53ae41691fdad2c48f"
server_key_version: "encryptionkey@0"
I20260808 22:34:15.796826 23081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:34:15.806257 23081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:34:15.808748 23081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:34:15.808884 23081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:34:15.809415 23081 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:34:15.810134 23081 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:34:15.810179 23081 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:15.810215 23081 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:34:15.810235 23081 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:15.818814 23081 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.130:41469
I20260808 22:34:15.818859 23279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.130:41469 every 8 connection(s)
I20260808 22:34:15.819731 23081 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
I20260808 22:34:15.823273 23282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:15.821681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.130:51607)
0808 22:34:15.822155 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:15.822578 (+ 423us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":112,"threads_started":1}
W20260808 22:34:15.823792 23280 heartbeater.cc:644] Failed to heartbeat to 127.31.87.190:41657 (0 consecutive failures): Network error: Failed to ping master at 127.31.87.190:41657: Client connection negotiation failed: client connection to 127.31.87.190:41657: connect: Connection refused (error 111)
I20260808 22:34:15.824321 23282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:15.824120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.130:42237)
0808 22:34:15.824215 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:15.824265 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 22:34:15.824833 23282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:15.824657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.130:60335)
0808 22:34:15.824740 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:15.824782 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 22:34:15.824970 23280 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 22:34:15.827834 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 23081
I20260808 22:34:15.828226 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.131:44849
--local_ip_for_outbound_sockets=127.31.87.131
--tserver_master_addrs=127.31.87.190:41657
--webserver_port=42581
--webserver_interface=127.31.87.131
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab
--principal=kudu/127.31.87.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:43005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:34:15.937252 23283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:34:15.937476 23283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:34:15.937532 23283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:34:15.941347 23283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:34:15.941393 23283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:34:15.941469 23283 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.131
I20260808 22:34:15.946323 23283 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:43005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab
--principal=kudu/127.31.87.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.131:44849
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.87.131
--webserver_port=42581
--tserver_master_addrs=127.31.87.190:41657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23283
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.131
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:34:15.947199 23283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:34:15.948087 23283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:34:15.956919 23294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:34:15.956930 23292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:34:15.957010 23283 server_base.cc:1034] running on GCE node
W20260808 22:34:15.956918 23296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:34:15.957497 23283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:34:15.958042 23283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:34:15.959198 23283 hybrid_clock.cc:648] HybridClock initialized: now 1786228455959171 us; error 43 us; skew 500 ppm
Aug 08 22:34:15 dist-test-slave-266d krb5kdc[21091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228455, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:34:15.961707 23283 init.cc:377] Logged in from keytab as kudu/127.31.87.131@KRBTEST.COM (short username kudu)
I20260808 22:34:15.962455 23283 webserver.cc:460] Webserver started at http://127.31.87.131:42581/ using document root <none> and password file <none>
I20260808 22:34:15.962931 23283 fs_manager.cc:359] Metadata directory not provided
I20260808 22:34:15.962986 23283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:34:15.966466 23283 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:34:15.968364 23304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:15.969487 23283 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:34:15.969585 23283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-2/wal
uuid: "d686840dc3584fbd9b3c7accdb6e26a3"
format_stamp: "Formatted at 2026-08-08 22:34:13 on dist-test-slave-266d"
server_key: "a7c67bcb19edb7a91b0d5b9b7bc6a0af"
server_key_iv: "481226f7429a5f096fd39bc0f0c6853b"
server_key_version: "encryptionkey@0"
I20260808 22:34:15.970042 23283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:34:16.006666 23283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:34:16.009719 23283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:34:16.009866 23283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:34:16.010411 23283 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:34:16.011199 23283 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:34:16.011245 23283 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:16.011282 23283 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:34:16.011303 23283 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:16.019407 23283 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.131:44849
I20260808 22:34:16.019466 23487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.131:44849 every 8 connection(s)
I20260808 22:34:16.020354 23283 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
I20260808 22:34:16.023819 23491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:16.022299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.131:54713)
0808 22:34:16.022744 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:16.023150 (+ 406us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":110,"threads_started":1}
W20260808 22:34:16.024214 23489 heartbeater.cc:644] Failed to heartbeat to 127.31.87.190:41657 (0 consecutive failures): Network error: Failed to ping master at 127.31.87.190:41657: Client connection negotiation failed: client connection to 127.31.87.190:41657: connect: Connection refused (error 111)
I20260808 22:34:16.024775 23491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:16.024566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.131:58527)
0808 22:34:16.024678 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:16.024723 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 22:34:16.024812 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 23283
I20260808 22:34:16.025403 23491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:16.025183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.131:37331)
0808 22:34:16.025292 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:16.025341 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 22:34:16.025592 23489 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 22:34:16.699232 23495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:16.698756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.129:36743)
0808 22:34:16.699030 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:16.699094 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":73,"threads_started":1}
I20260808 22:34:16.825785 23496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:16.825421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.130:52141)
0808 22:34:16.825638 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:16.825689 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":70,"threads_started":1}
I20260808 22:34:17.026422 23497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:17.026030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.131:34703)
0808 22:34:17.026262 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:17.026316 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":61,"threads_started":1}
I20260808 22:34:17.501900 23498 raft_consensus.cc:482] T 37cf708f0ea34e90b80755bdbe7d8489 P 4355aa00a45b496793ff2602e299e94e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:34:17.502100 23498 raft_consensus.cc:504] T 37cf708f0ea34e90b80755bdbe7d8489 P 4355aa00a45b496793ff2602e299e94e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d686840dc3584fbd9b3c7accdb6e26a3" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 44849 } } peers { permanent_uuid: "0db1fc6a077a4abb8d1650a2b9540d75" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41469 } } peers { permanent_uuid: "4355aa00a45b496793ff2602e299e94e" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 38117 } }
I20260808 22:34:17.503171 23498 leader_election.cc:288] T 37cf708f0ea34e90b80755bdbe7d8489 P 4355aa00a45b496793ff2602e299e94e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d686840dc3584fbd9b3c7accdb6e26a3 (127.31.87.131:44849), 0db1fc6a077a4abb8d1650a2b9540d75 (127.31.87.130:41469)
Aug 08 22:34:17 dist-test-slave-266d krb5kdc[21091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228455, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.129@KRBTEST.COM for kudu/127.31.87.130@KRBTEST.COM
Aug 08 22:34:17 dist-test-slave-266d krb5kdc[21091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228455, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.129@KRBTEST.COM for kudu/127.31.87.131@KRBTEST.COM
I20260808 22:34:17.517127 23499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:17.503372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.130:41469 (local address 127.31.87.129:37047)
0808 22:34:17.503656 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:17.503670 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:34:17.503776 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:34:17.505780 (+ 2004us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:34:17.505785 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:34:17.506149 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:34:17.506615 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:17.506628 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:17.507889 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:17.507893 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:17.508998 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:17.509008 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:17.509697 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:17.509701 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:17.509880 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:17.510518 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:34:17.510534 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:34:17.512250 (+ 1716us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:34:17.514423 (+ 2173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:17.514428 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:17.514440 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:17.514783 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:17.515150 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:17.515153 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:17.515155 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:17.515353 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:17.516079 (+ 726us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:34:17.516086 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:34:17.516461 (+ 375us) client_negotiation.cc:769] Sending connection context
0808 22:34:17.516677 (+ 216us) client_negotiation.cc:240] Negotiation successful
0808 22:34:17.516963 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"mutex_wait_us":148,"spinlock_wait_cycles":20352,"thread_start_us":72,"threads_started":1}
I20260808 22:34:17.517325 23502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:17.503517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:32819 (local address 127.31.87.131:44849)
0808 22:34:17.503862 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:17.503889 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:17.504065 (+ 176us) server_negotiation.cc:407] Connection header received
0808 22:34:17.504934 (+ 869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:17.504941 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:17.505409 (+ 468us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:17.505739 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:34:17.506801 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:17.507744 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:17.509134 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:17.509587 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:17.509735 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:17.512772 (+ 3037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:34:17.512779 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:34:17.512793 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:34:17.512825 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:34:17.514494 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:17.514916 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:17.514921 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:17.514925 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:17.515058 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:17.515447 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:17.515450 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:17.515452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:17.515961 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:34:17.516086 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:17.516856 (+ 770us) server_negotiation.cc:299] Negotiation successful
0808 22:34:17.517145 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":97,"threads_started":1}
I20260808 22:34:17.517329 23504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:17.503711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:37047 (local address 127.31.87.130:41469)
0808 22:34:17.504031 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:17.504042 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:17.504075 (+ 33us) server_negotiation.cc:407] Connection header received
0808 22:34:17.504929 (+ 854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:17.504937 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:17.505409 (+ 472us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:17.505739 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:34:17.506778 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:17.507758 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:17.509128 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:17.509580 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:17.509732 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:17.512446 (+ 2714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:34:17.512451 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:34:17.512462 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:34:17.512488 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:34:17.514311 (+ 1823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:17.514891 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:17.514897 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:17.514903 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:17.515058 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:17.515447 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:17.515450 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:17.515452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:17.515961 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:34:17.516086 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:17.516830 (+ 744us) server_negotiation.cc:299] Negotiation successful
0808 22:34:17.517145 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":127,"threads_started":1}
I20260808 22:34:17.517504 23500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:17.503372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.131:44849 (local address 127.31.87.129:32819)
0808 22:34:17.503955 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:17.503967 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:34:17.504065 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:34:17.506207 (+ 2142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:34:17.506211 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:34:17.506374 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:34:17.506615 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:17.506628 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:17.507936 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:17.507939 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:17.509000 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:17.509010 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:17.509701 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:17.509705 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:17.509878 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:17.510565 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:34:17.510577 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:34:17.512585 (+ 2008us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:34:17.514588 (+ 2003us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:17.514591 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:17.514593 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:17.514835 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:17.515150 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:17.515153 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:17.515155 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:17.515353 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:17.516079 (+ 726us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:34:17.516086 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:34:17.516461 (+ 375us) client_negotiation.cc:769] Sending connection context
0808 22:34:17.516677 (+ 216us) client_negotiation.cc:240] Negotiation successful
0808 22:34:17.517354 (+ 677us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"mutex_wait_us":717,"spinlock_wait_cycles":15104,"thread_start_us":73,"threads_started":1}
W20260808 22:34:17.519753 22911 leader_election.cc:341] T 37cf708f0ea34e90b80755bdbe7d8489 P 4355aa00a45b496793ff2602e299e94e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d686840dc3584fbd9b3c7accdb6e26a3 (127.31.87.131:44849): Not found: Tablet not found: 37cf708f0ea34e90b80755bdbe7d8489
W20260808 22:34:17.519899 22910 leader_election.cc:341] T 37cf708f0ea34e90b80755bdbe7d8489 P 4355aa00a45b496793ff2602e299e94e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0db1fc6a077a4abb8d1650a2b9540d75 (127.31.87.130:41469): Not found: Tablet not found: 37cf708f0ea34e90b80755bdbe7d8489
I20260808 22:34:17.519992 22910 leader_election.cc:302] T 37cf708f0ea34e90b80755bdbe7d8489 P 4355aa00a45b496793ff2602e299e94e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4355aa00a45b496793ff2602e299e94e; no voters: 0db1fc6a077a4abb8d1650a2b9540d75, d686840dc3584fbd9b3c7accdb6e26a3
I20260808 22:34:17.520226 23498 raft_consensus.cc:2726] T 37cf708f0ea34e90b80755bdbe7d8489 P 4355aa00a45b496793ff2602e299e94e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 22:34:17.700234 23500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:17.699967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.129:44503)
0808 22:34:17.700089 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:17.700146 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 22:34:17.826871 23505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:17.826464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.130:39117)
0808 22:34:17.826693 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:17.826750 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":76,"threads_started":1}
I20260808 22:34:18.027420 23506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:18.027030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.131:39549)
0808 22:34:18.027270 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:18.027328 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":70,"threads_started":1}
I20260808 22:34:18.701293 23507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:18.700857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.129:54579)
0808 22:34:18.701136 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:18.701192 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":72,"threads_started":1}
I20260808 22:34:18.827881 23508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:18.827505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.130:48787)
0808 22:34:18.827738 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:18.827790 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":74,"threads_started":1}
I20260808 22:34:19.028558 23509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:19.028105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.131:39409)
0808 22:34:19.028367 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:19.028436 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":73,"threads_started":1}
I20260808 22:34:19.107998 23511 raft_consensus.cc:482] T 37cf708f0ea34e90b80755bdbe7d8489 P 4355aa00a45b496793ff2602e299e94e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:34:19.108141 23511 raft_consensus.cc:504] T 37cf708f0ea34e90b80755bdbe7d8489 P 4355aa00a45b496793ff2602e299e94e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d686840dc3584fbd9b3c7accdb6e26a3" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 44849 } } peers { permanent_uuid: "0db1fc6a077a4abb8d1650a2b9540d75" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41469 } } peers { permanent_uuid: "4355aa00a45b496793ff2602e299e94e" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 38117 } }
I20260808 22:34:19.108577 23511 leader_election.cc:288] T 37cf708f0ea34e90b80755bdbe7d8489 P 4355aa00a45b496793ff2602e299e94e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d686840dc3584fbd9b3c7accdb6e26a3 (127.31.87.131:44849), 0db1fc6a077a4abb8d1650a2b9540d75 (127.31.87.130:41469)
W20260808 22:34:19.109356 22910 leader_election.cc:341] T 37cf708f0ea34e90b80755bdbe7d8489 P 4355aa00a45b496793ff2602e299e94e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0db1fc6a077a4abb8d1650a2b9540d75 (127.31.87.130:41469): Not found: Tablet not found: 37cf708f0ea34e90b80755bdbe7d8489
W20260808 22:34:19.109465 22911 leader_election.cc:341] T 37cf708f0ea34e90b80755bdbe7d8489 P 4355aa00a45b496793ff2602e299e94e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d686840dc3584fbd9b3c7accdb6e26a3 (127.31.87.131:44849): Not found: Tablet not found: 37cf708f0ea34e90b80755bdbe7d8489
I20260808 22:34:19.109555 22911 leader_election.cc:302] T 37cf708f0ea34e90b80755bdbe7d8489 P 4355aa00a45b496793ff2602e299e94e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4355aa00a45b496793ff2602e299e94e; no voters: 0db1fc6a077a4abb8d1650a2b9540d75, d686840dc3584fbd9b3c7accdb6e26a3
I20260808 22:34:19.109687 23511 raft_consensus.cc:2726] T 37cf708f0ea34e90b80755bdbe7d8489 P 4355aa00a45b496793ff2602e299e94e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 22:34:19.702358 23512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:19.701971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.129:37463)
0808 22:34:19.702197 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:19.702250 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":79,"threads_started":1}
I20260808 22:34:19.828879 23513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:19.828497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.130:49073)
0808 22:34:19.828742 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:19.828794 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":88,"threads_started":1}
I20260808 22:34:20.029627 23514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:20.029228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.131:50901)
0808 22:34:20.029477 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:20.029530 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":89,"threads_started":1}
I20260808 22:34:20.703610 23515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:20.703185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.129:53739)
0808 22:34:20.703463 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:20.703514 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":73,"threads_started":1}
I20260808 22:34:20.829877 23517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:20.829485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.130:46603)
0808 22:34:20.829739 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:20.829790 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":89,"threads_started":1}
I20260808 22:34:20.909494 23519 raft_consensus.cc:482] T 37cf708f0ea34e90b80755bdbe7d8489 P 4355aa00a45b496793ff2602e299e94e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:34:20.909631 23519 raft_consensus.cc:504] T 37cf708f0ea34e90b80755bdbe7d8489 P 4355aa00a45b496793ff2602e299e94e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d686840dc3584fbd9b3c7accdb6e26a3" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 44849 } } peers { permanent_uuid: "0db1fc6a077a4abb8d1650a2b9540d75" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41469 } } peers { permanent_uuid: "4355aa00a45b496793ff2602e299e94e" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 38117 } }
I20260808 22:34:20.909987 23519 leader_election.cc:288] T 37cf708f0ea34e90b80755bdbe7d8489 P 4355aa00a45b496793ff2602e299e94e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d686840dc3584fbd9b3c7accdb6e26a3 (127.31.87.131:44849), 0db1fc6a077a4abb8d1650a2b9540d75 (127.31.87.130:41469)
W20260808 22:34:20.910705 22910 leader_election.cc:341] T 37cf708f0ea34e90b80755bdbe7d8489 P 4355aa00a45b496793ff2602e299e94e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0db1fc6a077a4abb8d1650a2b9540d75 (127.31.87.130:41469): Not found: Tablet not found: 37cf708f0ea34e90b80755bdbe7d8489
W20260808 22:34:20.910831 22911 leader_election.cc:341] T 37cf708f0ea34e90b80755bdbe7d8489 P 4355aa00a45b496793ff2602e299e94e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d686840dc3584fbd9b3c7accdb6e26a3 (127.31.87.131:44849): Not found: Tablet not found: 37cf708f0ea34e90b80755bdbe7d8489
I20260808 22:34:20.910894 22911 leader_election.cc:302] T 37cf708f0ea34e90b80755bdbe7d8489 P 4355aa00a45b496793ff2602e299e94e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4355aa00a45b496793ff2602e299e94e; no voters: 0db1fc6a077a4abb8d1650a2b9540d75, d686840dc3584fbd9b3c7accdb6e26a3
I20260808 22:34:20.911024 23519 raft_consensus.cc:2726] T 37cf708f0ea34e90b80755bdbe7d8489 P 4355aa00a45b496793ff2602e299e94e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 22:34:21.030661 23524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:21.030280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.131:46547)
0808 22:34:21.030512 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:21.030567 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":72,"threads_started":1}
I20260808 22:34:21.704898 23525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:21.704306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.129:55955)
0808 22:34:21.704711 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:21.704779 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":83,"threads_started":1}
I20260808 22:34:21.830937 23526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:21.830530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.130:60525)
0808 22:34:21.830756 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:21.830822 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":70,"threads_started":1}
I20260808 22:34:22.031701 23527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:22.031321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.131:46017)
0808 22:34:22.031555 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:22.031611 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":71,"threads_started":1}
I20260808 22:34:22.705973 23528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:22.705599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.129:56931)
0808 22:34:22.705822 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:22.705875 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":76,"threads_started":1}
I20260808 22:34:22.832005 23529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:22.831559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.130:52695)
0808 22:34:22.831856 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:22.831911 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":72,"threads_started":1}
I20260808 22:34:23.032838 23530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:23.032394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.131:46551)
0808 22:34:23.032685 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:23.032742 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":69,"threads_started":1}
I20260808 22:34:23.707051 23531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:23.706658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.129:52377)
0808 22:34:23.706901 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:23.706954 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":75,"threads_started":1}
I20260808 22:34:23.833069 23533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:23.832652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.130:56405)
0808 22:34:23.832920 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:23.832973 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":85,"threads_started":1}
I20260808 22:34:24.033906 23535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:24.033474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.131:39543)
0808 22:34:24.033749 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:24.033809 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":92,"threads_started":1}
I20260808 22:34:24.708196 23538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:24.707750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.129:57911)
0808 22:34:24.708043 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:24.708101 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":103,"threads_started":1}
I20260808 22:34:24.834138 23539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:24.833712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.130:49971)
0808 22:34:24.833975 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:24.834030 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":91,"threads_started":1}
I20260808 22:34:25.034929 23541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:25.034552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.131:44905)
0808 22:34:25.034784 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:25.034839 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":73,"threads_started":1}
I20260808 22:34:25.438654 23544 raft_consensus.cc:482] T 37cf708f0ea34e90b80755bdbe7d8489 P 4355aa00a45b496793ff2602e299e94e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:34:25.438791 23544 raft_consensus.cc:504] T 37cf708f0ea34e90b80755bdbe7d8489 P 4355aa00a45b496793ff2602e299e94e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d686840dc3584fbd9b3c7accdb6e26a3" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 44849 } } peers { permanent_uuid: "0db1fc6a077a4abb8d1650a2b9540d75" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 41469 } } peers { permanent_uuid: "4355aa00a45b496793ff2602e299e94e" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 38117 } }
I20260808 22:34:25.439141 23544 leader_election.cc:288] T 37cf708f0ea34e90b80755bdbe7d8489 P 4355aa00a45b496793ff2602e299e94e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d686840dc3584fbd9b3c7accdb6e26a3 (127.31.87.131:44849), 0db1fc6a077a4abb8d1650a2b9540d75 (127.31.87.130:41469)
W20260808 22:34:25.439936 22910 leader_election.cc:341] T 37cf708f0ea34e90b80755bdbe7d8489 P 4355aa00a45b496793ff2602e299e94e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0db1fc6a077a4abb8d1650a2b9540d75 (127.31.87.130:41469): Not found: Tablet not found: 37cf708f0ea34e90b80755bdbe7d8489
W20260808 22:34:25.440062 22911 leader_election.cc:341] T 37cf708f0ea34e90b80755bdbe7d8489 P 4355aa00a45b496793ff2602e299e94e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d686840dc3584fbd9b3c7accdb6e26a3 (127.31.87.131:44849): Not found: Tablet not found: 37cf708f0ea34e90b80755bdbe7d8489
I20260808 22:34:25.440138 22911 leader_election.cc:302] T 37cf708f0ea34e90b80755bdbe7d8489 P 4355aa00a45b496793ff2602e299e94e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4355aa00a45b496793ff2602e299e94e; no voters: 0db1fc6a077a4abb8d1650a2b9540d75, d686840dc3584fbd9b3c7accdb6e26a3
I20260808 22:34:25.440274 23544 raft_consensus.cc:2726] T 37cf708f0ea34e90b80755bdbe7d8489 P 4355aa00a45b496793ff2602e299e94e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 22:34:25.709405 23545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:25.708950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.129:44131)
0808 22:34:25.709247 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:25.709306 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":75,"threads_started":1}
I20260808 22:34:25.835197 23546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:25.834798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.130:55873)
0808 22:34:25.835050 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:25.835105 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":74,"threads_started":1}
I20260808 22:34:26.036046 23549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:26.035624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.131:47505)
0808 22:34:26.035886 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:26.035948 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":88,"threads_started":1}
I20260808 22:34:26.710508 23550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:26.710122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.129:58723)
0808 22:34:26.710354 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:26.710409 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":73,"threads_started":1}
I20260808 22:34:26.836187 23552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:26.835834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.130:50621)
0808 22:34:26.836047 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:26.836096 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":72,"threads_started":1}
I20260808 22:34:27.037154 23553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:27.036712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.131:51485)
0808 22:34:27.036949 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:27.037010 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":70,"threads_started":1}
I20260808 22:34:27.711613 23554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:27.711185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.129:44525)
0808 22:34:27.711459 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:27.711514 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":98,"threads_started":1}
I20260808 22:34:27.837210 23556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:27.836809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.130:42447)
0808 22:34:27.837046 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:27.837119 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":86,"threads_started":1}
I20260808 22:34:28.038329 23557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:28.037914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.131:38289)
0808 22:34:28.038169 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:28.038232 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":66,"threads_started":1}
I20260808 22:34:28.712668 23558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:28.712272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.129:46793)
0808 22:34:28.712520 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:28.712573 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":82,"threads_started":1}
I20260808 22:34:28.838248 23559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:28.837838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.130:53283)
0808 22:34:28.838100 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:28.838155 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":81,"threads_started":1}
I20260808 22:34:29.039431 23564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:29.039029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.131:35313)
0808 22:34:29.039280 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:29.039336 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":69,"threads_started":1}
I20260808 22:34:29.713745 23565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:29.713329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.129:58029)
0808 22:34:29.713587 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:29.713647 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":79,"threads_started":1}
I20260808 22:34:29.839407 23567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:29.838938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.130:36961)
0808 22:34:29.839227 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:29.839293 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":111,"threads_started":1}
I20260808 22:34:30.040536 23568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:30.040107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.131:50739)
0808 22:34:30.040390 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:30.040444 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":92,"threads_started":1}
I20260808 22:34:30.715044 23569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:30.714639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.129:58977)
0808 22:34:30.714892 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:30.714945 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":77,"threads_started":1}
I20260808 22:34:30.840595 23573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:30.840198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.130:44991)
0808 22:34:30.840447 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:30.840499 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":74,"threads_started":1}
I20260808 22:34:31.030900 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 22844
I20260808 22:34:31.041805 23575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:31.041416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:41657 (local address 127.31.87.131:35331)
0808 22:34:31.041656 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:31.041710 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:41657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":66,"threads_started":1}
I20260808 22:34:31.046020 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 23081
I20260808 22:34:31.058993 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 23283
2026-08-08T22:34:31Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19543 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:34:31 dist-test-slave-266d krb5kdc[23581](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:34:31 dist-test-slave-266d krb5kdc[23581](info): set up 2 sockets
Aug 08 22:34:31 dist-test-slave-266d krb5kdc[23581](info): commencing operation
krb5kdc: starting...
W20260808 22:34:32.206980 32094 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Aug 08 22:34:32 dist-test-slave-266d krb5kdc[23581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228472, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:34:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:34:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.87.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
I20260808 22:34:32.321848 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:37545
--webserver_interface=127.31.87.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:34763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:37545
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:34:32.432245 23604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:34:32.432494 23604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:34:32.432533 23604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:34:32.436414 23604 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 22:34:32.436462 23604 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 22:34:32.436477 23604 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 22:34:32.436489 23604 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:34:32.436506 23604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:34:32.436527 23604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:34:32.436542 23604 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:34:32.436561 23604 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:34:32.441644 23604 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:34763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:37545
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:37545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.87.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23604
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:34:32.442577 23604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:34:32.443521 23604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:34:32.452291 23619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:34:32.452267 23615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:34:32.452317 23604 server_base.cc:1034] running on GCE node
W20260808 22:34:32.452258 23616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:34:32.453042 23604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:34:32.454103 23604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:34:32.455251 23604 hybrid_clock.cc:648] HybridClock initialized: now 1786228472455228 us; error 38 us; skew 500 ppm
Aug 08 22:34:32 dist-test-slave-266d krb5kdc[23581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228472, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:34:32.457839 23604 init.cc:377] Logged in from keytab as kudu/127.31.87.190@KRBTEST.COM (short username kudu)
I20260808 22:34:32.458679 23604 webserver.cc:460] Webserver started at http://127.31.87.190:43477/ using document root <none> and password file <none>
I20260808 22:34:32.459364 23604 fs_manager.cc:359] Metadata directory not provided
I20260808 22:34:32.459455 23604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:34:32.459721 23604 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:34:32.461781 23604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/master-0/data/instance:
uuid: "5c4f0c26263f4a79994b0c236d72fad8"
format_stamp: "Formatted at 2026-08-08 22:34:32 on dist-test-slave-266d"
server_key: "f44ab1ebc0fd14d0def43d3ad5410ed0"
server_key_iv: "f97f82f6979387d4b1fdfa875ec3a870"
server_key_version: "encryptionkey@0"
I20260808 22:34:32.462306 23604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/master-0/wal/instance:
uuid: "5c4f0c26263f4a79994b0c236d72fad8"
format_stamp: "Formatted at 2026-08-08 22:34:32 on dist-test-slave-266d"
server_key: "f44ab1ebc0fd14d0def43d3ad5410ed0"
server_key_iv: "f97f82f6979387d4b1fdfa875ec3a870"
server_key_version: "encryptionkey@0"
I20260808 22:34:32.465786 23604 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 22:34:32.467979 23631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:32.469131 23604 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:34:32.469249 23604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "5c4f0c26263f4a79994b0c236d72fad8"
format_stamp: "Formatted at 2026-08-08 22:34:32 on dist-test-slave-266d"
server_key: "f44ab1ebc0fd14d0def43d3ad5410ed0"
server_key_iv: "f97f82f6979387d4b1fdfa875ec3a870"
server_key_version: "encryptionkey@0"
I20260808 22:34:32.469342 23604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:34:32.487622 23604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:34:32.490206 23604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:34:32.490371 23604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:34:32.497051 23604 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.190:37545
I20260808 22:34:32.497323 23722 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.190:37545 every 8 connection(s)
I20260808 22:34:32.497943 23604 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:34:32.500537 23726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:34:32.505800 23726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8: Bootstrap starting.
I20260808 22:34:32.507191 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 23604
I20260808 22:34:32.507324 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/master-0/wal/instance
I20260808 22:34:32.507645 32094 external_mini_cluster.cc:1428] Setting key de609bc1ead73efaf4de1710ff6b24fa
I20260808 22:34:32.508507 23726 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8: Neither blocks nor log segments found. Creating new log.
I20260808 22:34:32.509356 23726 log.cc:824] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8: Log is configured to *not* fsync() on all Append() calls
I20260808 22:34:32.511384 23726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8: No bootstrap required, opened a new log
I20260808 22:34:32.515465 23726 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c4f0c26263f4a79994b0c236d72fad8" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 37545 } }
I20260808 22:34:32.515722 23726 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:34:32.515785 23726 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c4f0c26263f4a79994b0c236d72fad8, State: Initialized, Role: FOLLOWER
Aug 08 22:34:32 dist-test-slave-266d krb5kdc[23581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228472, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:34:32.516175 23726 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c4f0c26263f4a79994b0c236d72fad8" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 37545 } }
I20260808 22:34:32.516348 23726 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:34:32.516409 23726 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:34:32.516505 23726 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:34:32.517540 23726 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c4f0c26263f4a79994b0c236d72fad8" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 37545 } }
I20260808 22:34:32.517863 23726 leader_election.cc:302] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c4f0c26263f4a79994b0c236d72fad8; no voters:
I20260808 22:34:32.518142 23726 leader_election.cc:288] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:34:32.518251 23732 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:34:32.518466 23732 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8 [term 1 LEADER]: Becoming Leader. State: Replica: 5c4f0c26263f4a79994b0c236d72fad8, State: Running, Role: LEADER
I20260808 22:34:32.518831 23732 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c4f0c26263f4a79994b0c236d72fad8" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 37545 } }
I20260808 22:34:32.519220 23726 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:34:32.520352 23735 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c4f0c26263f4a79994b0c236d72fad8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c4f0c26263f4a79994b0c236d72fad8" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 37545 } } }
I20260808 22:34:32.520466 23735 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8 [sys.catalog]: This master's current role is: LEADER
I20260808 22:34:32.520757 23734 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c4f0c26263f4a79994b0c236d72fad8. Latest consensus state: current_term: 1 leader_uuid: "5c4f0c26263f4a79994b0c236d72fad8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c4f0c26263f4a79994b0c236d72fad8" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 37545 } } }
I20260808 22:34:32.520910 23734 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8 [sys.catalog]: This master's current role is: LEADER
I20260808 22:34:32.522176 23730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:32.508715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38234 (local address 127.31.87.190:37545)
0808 22:34:32.509109 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:32.509124 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:32.509159 (+ 35us) server_negotiation.cc:407] Connection header received
0808 22:34:32.509828 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:32.509851 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:32.510210 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:32.510482 (+ 272us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:34:32.511326 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:32.512343 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:32.513495 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:32.513945 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:32.514054 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:32.516755 (+ 2701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:34:32.516761 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:34:32.516777 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:34:32.516806 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:34:32.518510 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:32.519023 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:32.519029 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:32.519034 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:32.519164 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:32.519501 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:32.519504 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:32.519506 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:32.520012 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:34:32.520210 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:32.521060 (+ 850us) server_negotiation.cc:299] Negotiation successful
0808 22:34:32.521378 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":100,"threads_started":1}
W20260808 22:34:32.524765 23757 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:34:32.524853 23757 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:34:32.524907 23749 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:34:32.525749 23749 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:34:32.531194 23749 catalog_manager.cc:1347] Generated new cluster ID: d66d2e3ed939478ba6705e1ab06307ff
I20260808 22:34:32.531244 23749 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:34:32.545079 23749 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:34:32.546371 23749 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:34:32.563750 23749 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8: Generated new TSK 0
I20260808 22:34:32.564358 23749 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.87.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab.
Entry for principal kudu/127.31.87.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab.
I20260808 22:34:32.593783 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.129:0
--local_ip_for_outbound_sockets=127.31.87.129
--webserver_interface=127.31.87.129
--webserver_port=0
--tserver_master_addrs=127.31.87.190:37545
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab
--principal=kudu/127.31.87.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:34763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:34:32.700838 23763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:34:32.701063 23763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:34:32.701136 23763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:34:32.704988 23763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:34:32.705039 23763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:34:32.705124 23763 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.129
I20260808 22:34:32.710130 23763 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:34763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab
--principal=kudu/127.31.87.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.87.129
--webserver_port=0
--tserver_master_addrs=127.31.87.190:37545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23763
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.129
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:34:32.711051 23763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:34:32.711993 23763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:34:32.720985 23770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:34:32.721005 23768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:34:32.720984 23774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:34:32.721176 23763 server_base.cc:1034] running on GCE node
I20260808 22:34:32.721773 23763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:34:32.722313 23763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:34:32.723474 23763 hybrid_clock.cc:648] HybridClock initialized: now 1786228472723456 us; error 27 us; skew 500 ppm
Aug 08 22:34:32 dist-test-slave-266d krb5kdc[23581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228472, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:34:32.726007 23763 init.cc:377] Logged in from keytab as kudu/127.31.87.129@KRBTEST.COM (short username kudu)
I20260808 22:34:32.726809 23763 webserver.cc:460] Webserver started at http://127.31.87.129:45475/ using document root <none> and password file <none>
I20260808 22:34:32.727312 23763 fs_manager.cc:359] Metadata directory not provided
I20260808 22:34:32.727376 23763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:34:32.727576 23763 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:34:32.729565 23763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-0/data/instance:
uuid: "b04503a3ef6e4d6e833b22c13c65122c"
format_stamp: "Formatted at 2026-08-08 22:34:32 on dist-test-slave-266d"
server_key: "d9262e12f89f8cf4cb3863fb25fd7135"
server_key_iv: "7fb3e8b41f829b8c20ae8a72976eefe1"
server_key_version: "encryptionkey@0"
I20260808 22:34:32.730087 23763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-0/wal/instance:
uuid: "b04503a3ef6e4d6e833b22c13c65122c"
format_stamp: "Formatted at 2026-08-08 22:34:32 on dist-test-slave-266d"
server_key: "d9262e12f89f8cf4cb3863fb25fd7135"
server_key_iv: "7fb3e8b41f829b8c20ae8a72976eefe1"
server_key_version: "encryptionkey@0"
I20260808 22:34:32.733505 23763 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 22:34:32.735672 23786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:32.736771 23763 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:34:32.736873 23763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-0/wal
uuid: "b04503a3ef6e4d6e833b22c13c65122c"
format_stamp: "Formatted at 2026-08-08 22:34:32 on dist-test-slave-266d"
server_key: "d9262e12f89f8cf4cb3863fb25fd7135"
server_key_iv: "7fb3e8b41f829b8c20ae8a72976eefe1"
server_key_version: "encryptionkey@0"
I20260808 22:34:32.736963 23763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:34:32.756513 23763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:34:32.759125 23763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:34:32.759286 23763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:34:32.759804 23763 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:34:32.760547 23763 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:34:32.760593 23763 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:32.760653 23763 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:34:32.760680 23763 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:32.768654 23763 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.129:43459
I20260808 22:34:32.768671 23948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.129:43459 every 8 connection(s)
I20260808 22:34:32.769503 23763 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
Aug 08 22:34:32 dist-test-slave-266d krb5kdc[23581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228472, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.129@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:34:32.779422 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 23763
I20260808 22:34:32.779525 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-0/wal/instance
I20260808 22:34:32.779806 32094 external_mini_cluster.cc:1428] Setting key f30c0438d2b5a6dee11249d10fd75b1f
I20260808 22:34:32.781785 23730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:32.771125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:56821 (local address 127.31.87.190:37545)
0808 22:34:32.771261 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:32.771265 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:32.772295 (+ 1030us) server_negotiation.cc:407] Connection header received
0808 22:34:32.773175 (+ 880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:32.773178 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:32.773226 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:32.773297 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:34:32.774366 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:32.774889 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:32.775921 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:32.776150 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:32.776222 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:32.778377 (+ 2155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:34:32.778380 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:34:32.778382 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:34:32.778406 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:34:32.779732 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:32.780270 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:32.780274 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:32.780275 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:32.780373 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:32.780721 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:32.780724 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:32.780726 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:32.780922 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:34:32.781016 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:32.781563 (+ 547us) server_negotiation.cc:299] Negotiation successful
0808 22:34:32.781667 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 22:34:32.782702 23952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:32.771498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.129:56821)
0808 22:34:32.772139 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:32.772191 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 22:34:32.772966 (+ 775us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:34:32.773466 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:34:32.773472 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:34:32.773800 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:34:32.774234 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:32.774247 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:32.774976 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:32.774979 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:32.775821 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:32.775829 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:32.776255 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:32.776257 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:32.776401 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:32.776863 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:34:32.776877 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:34:32.778235 (+ 1358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:34:32.779836 (+ 1601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:32.779842 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:32.779856 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:32.780167 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:32.780463 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:32.780466 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:32.780468 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:32.780638 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:32.781009 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:34:32.781014 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:34:32.781309 (+ 295us) client_negotiation.cc:769] Sending connection context
0808 22:34:32.781500 (+ 191us) client_negotiation.cc:240] Negotiation successful
0808 22:34:32.781816 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":118,"threads_started":1}
I20260808 22:34:32.784133 23949 heartbeater.cc:347] Connected to a master server at 127.31.87.190:37545
I20260808 22:34:32.784353 23949 heartbeater.cc:464] Registering TS with master...
I20260808 22:34:32.784791 23949 heartbeater.cc:511] Master 127.31.87.190:37545 requested a full tablet report, sending...
I20260808 22:34:32.786392 23655 ts_manager.cc:194] Registered new tserver with Master: b04503a3ef6e4d6e833b22c13c65122c (127.31.87.129:43459)
I20260808 22:34:32.787940 23655 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.129@KRBTEST.COM'} at 127.31.87.129:56821
WARNING: no policy specified for kudu/127.31.87.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab.
Entry for principal kudu/127.31.87.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab.
I20260808 22:34:32.807235 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.130:0
--local_ip_for_outbound_sockets=127.31.87.130
--webserver_interface=127.31.87.130
--webserver_port=0
--tserver_master_addrs=127.31.87.190:37545
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab
--principal=kudu/127.31.87.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:34763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:34:32.924124 23956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:34:32.924357 23956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:34:32.924415 23956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:34:32.928646 23956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:34:32.928699 23956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:34:32.928777 23956 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.130
I20260808 22:34:32.933766 23956 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:34763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab
--principal=kudu/127.31.87.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.87.130
--webserver_port=0
--tserver_master_addrs=127.31.87.190:37545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23956
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.130
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:34:32.934716 23956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:34:32.935668 23956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:34:32.944873 23964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:34:32.944875 23963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:34:32.944962 23956 server_base.cc:1034] running on GCE node
W20260808 22:34:32.944904 23966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:34:32.945446 23956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:34:32.946027 23956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:34:32.947176 23956 hybrid_clock.cc:648] HybridClock initialized: now 1786228472947158 us; error 29 us; skew 500 ppm
Aug 08 22:34:32 dist-test-slave-266d krb5kdc[23581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228472, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:34:32.949640 23956 init.cc:377] Logged in from keytab as kudu/127.31.87.130@KRBTEST.COM (short username kudu)
I20260808 22:34:32.950397 23956 webserver.cc:460] Webserver started at http://127.31.87.130:45801/ using document root <none> and password file <none>
I20260808 22:34:32.950897 23956 fs_manager.cc:359] Metadata directory not provided
I20260808 22:34:32.950961 23956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:34:32.951146 23956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:34:32.953059 23956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-1/data/instance:
uuid: "d868a18f455648d9b2d67353e0059e9a"
format_stamp: "Formatted at 2026-08-08 22:34:32 on dist-test-slave-266d"
server_key: "0a91ad264b934d578ca5f97b838f95fc"
server_key_iv: "a30caef0655e0ace7cde4c2a6f13e92a"
server_key_version: "encryptionkey@0"
I20260808 22:34:32.953603 23956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-1/wal/instance:
uuid: "d868a18f455648d9b2d67353e0059e9a"
format_stamp: "Formatted at 2026-08-08 22:34:32 on dist-test-slave-266d"
server_key: "0a91ad264b934d578ca5f97b838f95fc"
server_key_iv: "a30caef0655e0ace7cde4c2a6f13e92a"
server_key_version: "encryptionkey@0"
I20260808 22:34:32.956930 23956 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 22:34:32.959160 23973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:32.960238 23956 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:34:32.960346 23956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-1/wal
uuid: "d868a18f455648d9b2d67353e0059e9a"
format_stamp: "Formatted at 2026-08-08 22:34:32 on dist-test-slave-266d"
server_key: "0a91ad264b934d578ca5f97b838f95fc"
server_key_iv: "a30caef0655e0ace7cde4c2a6f13e92a"
server_key_version: "encryptionkey@0"
I20260808 22:34:32.960433 23956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:34:32.980073 23956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:34:32.982772 23956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:34:32.982939 23956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:34:32.983470 23956 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:34:32.984236 23956 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:34:32.984287 23956 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:32.984337 23956 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:34:32.984364 23956 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:32.992842 23956 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.130:40277
I20260808 22:34:32.992918 24155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.130:40277 every 8 connection(s)
I20260808 22:34:32.993690 23956 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
Aug 08 22:34:33 dist-test-slave-266d krb5kdc[23581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228472, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.130@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:34:33.002944 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 23956
I20260808 22:34:33.003079 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-1/wal/instance
I20260808 22:34:33.003432 32094 external_mini_cluster.cc:1428] Setting key 20bb870c61b9677da68fd351a9a5bfd6
I20260808 22:34:33.005954 23730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:32.995253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:51399 (local address 127.31.87.190:37545)
0808 22:34:32.995366 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:32.995370 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:32.996278 (+ 908us) server_negotiation.cc:407] Connection header received
0808 22:34:32.997163 (+ 885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:32.997167 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:32.997228 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:32.997303 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:34:32.998402 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:32.998930 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:33.000014 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:33.000233 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:33.000323 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:33.002539 (+ 2216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:34:33.002542 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:34:33.002544 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:34:33.002568 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:34:33.003998 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:33.004455 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:33.004459 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:33.004460 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:33.004541 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:33.004856 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:33.004860 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:33.004862 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:33.005068 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:34:33.005219 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:33.005669 (+ 450us) server_negotiation.cc:299] Negotiation successful
0808 22:34:33.005795 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 22:34:33.006722 24159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:32.995551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.130:51399)
0808 22:34:32.996143 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:32.996180 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 22:34:32.996945 (+ 765us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:34:32.997475 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:34:32.997482 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:34:32.997816 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:34:32.998260 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:32.998273 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:32.999047 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:32.999049 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:32.999896 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:32.999905 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:33.000372 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:33.000377 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:33.000530 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:33.000977 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:34:33.000993 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:34:33.002388 (+ 1395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:34:33.004100 (+ 1712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:33.004104 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:33.004115 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:33.004353 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:33.004622 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:33.004625 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:33.004627 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:33.004756 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:33.005218 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:34:33.005225 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:34:33.005453 (+ 228us) client_negotiation.cc:769] Sending connection context
0808 22:34:33.005613 (+ 160us) client_negotiation.cc:240] Negotiation successful
0808 22:34:33.005922 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":106,"threads_started":1}
I20260808 22:34:33.007858 24156 heartbeater.cc:347] Connected to a master server at 127.31.87.190:37545
I20260808 22:34:33.008065 24156 heartbeater.cc:464] Registering TS with master...
I20260808 22:34:33.008496 24156 heartbeater.cc:511] Master 127.31.87.190:37545 requested a full tablet report, sending...
I20260808 22:34:33.009522 23655 ts_manager.cc:194] Registered new tserver with Master: d868a18f455648d9b2d67353e0059e9a (127.31.87.130:40277)
I20260808 22:34:33.010079 23655 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.130@KRBTEST.COM'} at 127.31.87.130:51399
WARNING: no policy specified for kudu/127.31.87.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab.
Entry for principal kudu/127.31.87.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab.
I20260808 22:34:33.030288 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.131:0
--local_ip_for_outbound_sockets=127.31.87.131
--webserver_interface=127.31.87.131
--webserver_port=0
--tserver_master_addrs=127.31.87.190:37545
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab
--principal=kudu/127.31.87.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:34763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:34:33.137406 24163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:34:33.137635 24163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:34:33.137692 24163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:34:33.141656 24163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:34:33.141707 24163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:34:33.141784 24163 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.131
I20260808 22:34:33.146719 24163 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:34763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab
--principal=kudu/127.31.87.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.87.131
--webserver_port=0
--tserver_master_addrs=127.31.87.190:37545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24163
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.131
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:34:33.147612 24163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:34:33.148500 24163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:34:33.157298 24173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:34:33.157392 24175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:34:33.157372 24171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:34:33.157361 24163 server_base.cc:1034] running on GCE node
I20260808 22:34:33.157891 24163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:34:33.158463 24163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:34:33.159614 24163 hybrid_clock.cc:648] HybridClock initialized: now 1786228473159597 us; error 30 us; skew 500 ppm
Aug 08 22:34:33 dist-test-slave-266d krb5kdc[23581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228473, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:34:33.162132 24163 init.cc:377] Logged in from keytab as kudu/127.31.87.131@KRBTEST.COM (short username kudu)
I20260808 22:34:33.162909 24163 webserver.cc:460] Webserver started at http://127.31.87.131:35617/ using document root <none> and password file <none>
I20260808 22:34:33.163424 24163 fs_manager.cc:359] Metadata directory not provided
I20260808 22:34:33.163479 24163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:34:33.163640 24163 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:34:33.165553 24163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-2/data/instance:
uuid: "43172513b66542d79f1d327998eb886c"
format_stamp: "Formatted at 2026-08-08 22:34:33 on dist-test-slave-266d"
server_key: "b28fe866acfd0353c226c51e0002da45"
server_key_iv: "13c50f9463f0a73c1d1ed778c9b16ed4"
server_key_version: "encryptionkey@0"
I20260808 22:34:33.166029 24163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-2/wal/instance:
uuid: "43172513b66542d79f1d327998eb886c"
format_stamp: "Formatted at 2026-08-08 22:34:33 on dist-test-slave-266d"
server_key: "b28fe866acfd0353c226c51e0002da45"
server_key_iv: "13c50f9463f0a73c1d1ed778c9b16ed4"
server_key_version: "encryptionkey@0"
I20260808 22:34:33.169356 24163 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:34:33.171456 24183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:33.172590 24163 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:34:33.172677 24163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-2/wal
uuid: "43172513b66542d79f1d327998eb886c"
format_stamp: "Formatted at 2026-08-08 22:34:33 on dist-test-slave-266d"
server_key: "b28fe866acfd0353c226c51e0002da45"
server_key_iv: "13c50f9463f0a73c1d1ed778c9b16ed4"
server_key_version: "encryptionkey@0"
I20260808 22:34:33.172776 24163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:34:33.183971 24163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:34:33.186421 24163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:34:33.186553 24163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:34:33.187004 24163 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:34:33.187695 24163 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:34:33.187739 24163 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:33.187774 24163 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:34:33.187794 24163 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:33.196079 24163 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.131:36833
I20260808 22:34:33.196120 24364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.131:36833 every 8 connection(s)
I20260808 22:34:33.196887 24163 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
Aug 08 22:34:33 dist-test-slave-266d krb5kdc[23581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228473, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.131@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:34:33.205891 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 24163
I20260808 22:34:33.206009 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-2/wal/instance
I20260808 22:34:33.206327 32094 external_mini_cluster.cc:1428] Setting key 98a5c24c86d72979e80cef342a28f06f
I20260808 22:34:33.208972 23730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:33.198480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:49203 (local address 127.31.87.190:37545)
0808 22:34:33.198635 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:33.198640 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:33.199597 (+ 957us) server_negotiation.cc:407] Connection header received
0808 22:34:33.200427 (+ 830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:33.200429 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:33.200477 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:33.200536 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:34:33.201607 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:33.202123 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:33.203141 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:33.203347 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:33.203410 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:33.205578 (+ 2168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:34:33.205581 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:34:33.205583 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:34:33.205607 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:34:33.207070 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:33.207576 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:33.207580 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:33.207581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:33.207674 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:33.208018 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:33.208021 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:33.208022 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:33.208215 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:34:33.208303 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:33.208765 (+ 462us) server_negotiation.cc:299] Negotiation successful
0808 22:34:33.208858 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 22:34:33.209745 24370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:33.198823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.131:49203)
0808 22:34:33.199464 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:33.199502 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 22:34:33.200244 (+ 742us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:34:33.200695 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:34:33.200702 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:34:33.201030 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:34:33.201486 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:33.201498 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:33.202207 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:33.202209 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:33.203056 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:33.203064 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:33.203449 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:33.203451 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:33.203582 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:33.204076 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:34:33.204092 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:34:33.205418 (+ 1326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:34:33.207187 (+ 1769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:33.207193 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:33.207206 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:33.207486 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:33.207758 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:33.207761 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:33.207771 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:33.207933 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:33.208317 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:34:33.208323 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:34:33.208568 (+ 245us) client_negotiation.cc:769] Sending connection context
0808 22:34:33.208715 (+ 147us) client_negotiation.cc:240] Negotiation successful
0808 22:34:33.208945 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":136,"threads_started":1}
I20260808 22:34:33.210731 24365 heartbeater.cc:347] Connected to a master server at 127.31.87.190:37545
I20260808 22:34:33.210919 24365 heartbeater.cc:464] Registering TS with master...
I20260808 22:34:33.211309 24365 heartbeater.cc:511] Master 127.31.87.190:37545 requested a full tablet report, sending...
I20260808 22:34:33.212167 23655 ts_manager.cc:194] Registered new tserver with Master: 43172513b66542d79f1d327998eb886c (127.31.87.131:36833)
I20260808 22:34:33.212687 23655 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.131@KRBTEST.COM'} at 127.31.87.131:49203
I20260808 22:34:33.219844 32094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:34:33.227670 23730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:33.221727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38250 (local address 127.31.87.190:37545)
0808 22:34:33.221824 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:33.221827 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:33.221996 (+ 169us) server_negotiation.cc:407] Connection header received
0808 22:34:33.222088 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:33.222090 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:33.222122 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:33.222180 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:34:33.222747 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:33.223214 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:33.223958 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:33.224149 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:33.224211 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:33.225376 (+ 1165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:34:33.225380 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:34:33.225381 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:34:33.225402 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:34:33.226457 (+ 1055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:33.226807 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:33.226809 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:33.226810 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:33.226869 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:33.227041 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:33.227043 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:33.227044 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:33.227236 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:34:33.227339 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:33.227477 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 22:34:33.227562 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 22:34:33.230131 23655 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38250:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:34:33.232981 23655 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:34:33.246686 24382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:33.240891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.130:40277 (local address 127.0.0.1:51956)
0808 22:34:33.241308 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:33.241355 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 22:34:33.241470 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:34:33.242549 (+ 1079us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:34:33.242555 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:34:33.242577 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:34:33.242760 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:33.242768 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:33.243725 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:33.243728 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:33.244955 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:33.244967 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:33.246034 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:33.246037 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:33.246268 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:33.246303 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 22:34:33.246412 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 22:34:33.246544 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":66,"thread_start_us":71,"threads_started":1}
I20260808 22:34:33.246868 24387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:33.241785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58100 (local address 127.31.87.129:43459)
0808 22:34:33.242184 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:33.242199 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:33.242221 (+ 22us) server_negotiation.cc:407] Connection header received
0808 22:34:33.242295 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:33.242300 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:33.242498 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:33.242646 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:34:33.242928 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:33.243981 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:33.245504 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:33.246294 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:33.246419 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:33.246551 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:33.246644 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 22:34:33.246730 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":61,"threads_started":1}
I20260808 22:34:33.246857 24383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:33.241116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:43459 (local address 127.0.0.1:58100)
0808 22:34:33.241494 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:33.241507 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:34:33.241593 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:34:33.242629 (+ 1036us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:34:33.242631 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:34:33.242645 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:34:33.242791 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:33.242802 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:33.244106 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:33.244109 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:33.245387 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:33.245397 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:33.246416 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:33.246419 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:33.246519 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:33.246541 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 22:34:33.246617 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 22:34:33.246687 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"mutex_wait_us":36,"thread_start_us":93,"threads_started":1}
I20260808 22:34:33.247170 24385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:33.241016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51956 (local address 127.31.87.130:40277)
0808 22:34:33.242086 (+ 1070us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:33.242096 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:33.242124 (+ 28us) server_negotiation.cc:407] Connection header received
0808 22:34:33.242192 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:33.242197 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:33.242400 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:33.242551 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:34:33.242852 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:33.243582 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:33.245127 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:33.245898 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:33.246022 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:33.246167 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:33.246943 (+ 776us) server_negotiation.cc:299] Negotiation successful
0808 22:34:33.247039 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":965,"thread_start_us":193,"threads_started":1}
I20260808 22:34:33.247920 24386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:33.241616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.131:36833 (local address 127.0.0.1:46372)
0808 22:34:33.242886 (+ 1270us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:33.242898 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:34:33.242976 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:34:33.243611 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:34:33.243614 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:34:33.243634 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:34:33.243724 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:33.243730 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:33.244897 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:33.244900 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:33.246243 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:33.246256 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:33.247544 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:33.247548 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:33.247642 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:33.247665 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 22:34:33.247736 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 22:34:33.247802 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1212,"mutex_wait_us":61,"thread_start_us":104,"threads_started":1}
I20260808 22:34:33.248320 24389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:33.241768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46372 (local address 127.31.87.131:36833)
0808 22:34:33.243206 (+ 1438us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:33.243217 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:33.243237 (+ 20us) server_negotiation.cc:407] Connection header received
0808 22:34:33.243306 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:33.243311 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:33.243465 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:33.243605 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:34:33.243820 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:33.244745 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:33.246373 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:33.247165 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:33.247343 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:33.247480 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:33.248129 (+ 649us) server_negotiation.cc:299] Negotiation successful
0808 22:34:33.248214 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1339,"thread_start_us":93,"threads_started":1}
I20260808 22:34:33.250113 23854 tablet_service.cc:1467] Processing CreateTablet for tablet b786f0fbb7c849f1addecb41a24572b3 (DEFAULT_TABLE table=test-table [id=a4e97a00f7f140fd99b5139d6ac80816]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:34:33.250113 24058 tablet_service.cc:1467] Processing CreateTablet for tablet b786f0fbb7c849f1addecb41a24572b3 (DEFAULT_TABLE table=test-table [id=a4e97a00f7f140fd99b5139d6ac80816]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:34:33.250687 24265 tablet_service.cc:1467] Processing CreateTablet for tablet b786f0fbb7c849f1addecb41a24572b3 (DEFAULT_TABLE table=test-table [id=a4e97a00f7f140fd99b5139d6ac80816]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:34:33.251214 23854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b786f0fbb7c849f1addecb41a24572b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:34:33.251214 24058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b786f0fbb7c849f1addecb41a24572b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:34:33.251552 24265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b786f0fbb7c849f1addecb41a24572b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:34:33.258373 24391 tablet_bootstrap.cc:492] T b786f0fbb7c849f1addecb41a24572b3 P 43172513b66542d79f1d327998eb886c: Bootstrap starting.
I20260808 22:34:33.258602 24392 tablet_bootstrap.cc:492] T b786f0fbb7c849f1addecb41a24572b3 P d868a18f455648d9b2d67353e0059e9a: Bootstrap starting.
I20260808 22:34:33.258602 24393 tablet_bootstrap.cc:492] T b786f0fbb7c849f1addecb41a24572b3 P b04503a3ef6e4d6e833b22c13c65122c: Bootstrap starting.
I20260808 22:34:33.260391 24391 tablet_bootstrap.cc:654] T b786f0fbb7c849f1addecb41a24572b3 P 43172513b66542d79f1d327998eb886c: Neither blocks nor log segments found. Creating new log.
I20260808 22:34:33.260675 24393 tablet_bootstrap.cc:654] T b786f0fbb7c849f1addecb41a24572b3 P b04503a3ef6e4d6e833b22c13c65122c: Neither blocks nor log segments found. Creating new log.
I20260808 22:34:33.261114 24392 tablet_bootstrap.cc:654] T b786f0fbb7c849f1addecb41a24572b3 P d868a18f455648d9b2d67353e0059e9a: Neither blocks nor log segments found. Creating new log.
I20260808 22:34:33.261539 24391 log.cc:824] T b786f0fbb7c849f1addecb41a24572b3 P 43172513b66542d79f1d327998eb886c: Log is configured to *not* fsync() on all Append() calls
I20260808 22:34:33.261885 24392 log.cc:824] T b786f0fbb7c849f1addecb41a24572b3 P d868a18f455648d9b2d67353e0059e9a: Log is configured to *not* fsync() on all Append() calls
I20260808 22:34:33.261885 24393 log.cc:824] T b786f0fbb7c849f1addecb41a24572b3 P b04503a3ef6e4d6e833b22c13c65122c: Log is configured to *not* fsync() on all Append() calls
I20260808 22:34:33.263687 24392 tablet_bootstrap.cc:492] T b786f0fbb7c849f1addecb41a24572b3 P d868a18f455648d9b2d67353e0059e9a: No bootstrap required, opened a new log
I20260808 22:34:33.263708 24391 tablet_bootstrap.cc:492] T b786f0fbb7c849f1addecb41a24572b3 P 43172513b66542d79f1d327998eb886c: No bootstrap required, opened a new log
I20260808 22:34:33.263898 24391 ts_tablet_manager.cc:1397] T b786f0fbb7c849f1addecb41a24572b3 P 43172513b66542d79f1d327998eb886c: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 22:34:33.263916 24392 ts_tablet_manager.cc:1397] T b786f0fbb7c849f1addecb41a24572b3 P d868a18f455648d9b2d67353e0059e9a: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260808 22:34:33.264182 24393 tablet_bootstrap.cc:492] T b786f0fbb7c849f1addecb41a24572b3 P b04503a3ef6e4d6e833b22c13c65122c: No bootstrap required, opened a new log
I20260808 22:34:33.264432 24393 ts_tablet_manager.cc:1397] T b786f0fbb7c849f1addecb41a24572b3 P b04503a3ef6e4d6e833b22c13c65122c: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 22:34:33.266824 24391 raft_consensus.cc:348] T b786f0fbb7c849f1addecb41a24572b3 P 43172513b66542d79f1d327998eb886c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d868a18f455648d9b2d67353e0059e9a" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 40277 } } peers { permanent_uuid: "43172513b66542d79f1d327998eb886c" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36833 } } peers { permanent_uuid: "b04503a3ef6e4d6e833b22c13c65122c" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 43459 } }
I20260808 22:34:33.267045 24391 raft_consensus.cc:374] T b786f0fbb7c849f1addecb41a24572b3 P 43172513b66542d79f1d327998eb886c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:34:33.267118 24391 raft_consensus.cc:729] T b786f0fbb7c849f1addecb41a24572b3 P 43172513b66542d79f1d327998eb886c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43172513b66542d79f1d327998eb886c, State: Initialized, Role: FOLLOWER
I20260808 22:34:33.267527 24391 consensus_queue.cc:260] T b786f0fbb7c849f1addecb41a24572b3 P 43172513b66542d79f1d327998eb886c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d868a18f455648d9b2d67353e0059e9a" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 40277 } } peers { permanent_uuid: "43172513b66542d79f1d327998eb886c" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36833 } } peers { permanent_uuid: "b04503a3ef6e4d6e833b22c13c65122c" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 43459 } }
I20260808 22:34:33.267558 24392 raft_consensus.cc:348] T b786f0fbb7c849f1addecb41a24572b3 P d868a18f455648d9b2d67353e0059e9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d868a18f455648d9b2d67353e0059e9a" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 40277 } } peers { permanent_uuid: "43172513b66542d79f1d327998eb886c" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36833 } } peers { permanent_uuid: "b04503a3ef6e4d6e833b22c13c65122c" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 43459 } }
I20260808 22:34:33.267840 24392 raft_consensus.cc:374] T b786f0fbb7c849f1addecb41a24572b3 P d868a18f455648d9b2d67353e0059e9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:34:33.267912 24392 raft_consensus.cc:729] T b786f0fbb7c849f1addecb41a24572b3 P d868a18f455648d9b2d67353e0059e9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d868a18f455648d9b2d67353e0059e9a, State: Initialized, Role: FOLLOWER
I20260808 22:34:33.267956 24393 raft_consensus.cc:348] T b786f0fbb7c849f1addecb41a24572b3 P b04503a3ef6e4d6e833b22c13c65122c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d868a18f455648d9b2d67353e0059e9a" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 40277 } } peers { permanent_uuid: "43172513b66542d79f1d327998eb886c" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36833 } } peers { permanent_uuid: "b04503a3ef6e4d6e833b22c13c65122c" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 43459 } }
I20260808 22:34:33.268210 24393 raft_consensus.cc:374] T b786f0fbb7c849f1addecb41a24572b3 P b04503a3ef6e4d6e833b22c13c65122c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:34:33.268267 24393 raft_consensus.cc:729] T b786f0fbb7c849f1addecb41a24572b3 P b04503a3ef6e4d6e833b22c13c65122c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b04503a3ef6e4d6e833b22c13c65122c, State: Initialized, Role: FOLLOWER
I20260808 22:34:33.268323 24392 consensus_queue.cc:260] T b786f0fbb7c849f1addecb41a24572b3 P d868a18f455648d9b2d67353e0059e9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d868a18f455648d9b2d67353e0059e9a" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 40277 } } peers { permanent_uuid: "43172513b66542d79f1d327998eb886c" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36833 } } peers { permanent_uuid: "b04503a3ef6e4d6e833b22c13c65122c" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 43459 } }
I20260808 22:34:33.268446 24391 ts_tablet_manager.cc:1428] T b786f0fbb7c849f1addecb41a24572b3 P 43172513b66542d79f1d327998eb886c: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260808 22:34:33.268786 24365 heartbeater.cc:503] Master 127.31.87.190:37545 was elected leader, sending a full tablet report...
I20260808 22:34:33.268923 24156 heartbeater.cc:503] Master 127.31.87.190:37545 was elected leader, sending a full tablet report...
I20260808 22:34:33.269243 24392 ts_tablet_manager.cc:1428] T b786f0fbb7c849f1addecb41a24572b3 P d868a18f455648d9b2d67353e0059e9a: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 22:34:33.268697 24393 consensus_queue.cc:260] T b786f0fbb7c849f1addecb41a24572b3 P b04503a3ef6e4d6e833b22c13c65122c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d868a18f455648d9b2d67353e0059e9a" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 40277 } } peers { permanent_uuid: "43172513b66542d79f1d327998eb886c" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36833 } } peers { permanent_uuid: "b04503a3ef6e4d6e833b22c13c65122c" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 43459 } }
I20260808 22:34:33.270550 24393 ts_tablet_manager.cc:1428] T b786f0fbb7c849f1addecb41a24572b3 P b04503a3ef6e4d6e833b22c13c65122c: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260808 22:34:33.270658 23951 tablet.cc:2369] T b786f0fbb7c849f1addecb41a24572b3 P b04503a3ef6e4d6e833b22c13c65122c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:34:33.270905 23949 heartbeater.cc:503] Master 127.31.87.190:37545 was elected leader, sending a full tablet report...
I20260808 22:34:33.397226 24399 raft_consensus.cc:482] T b786f0fbb7c849f1addecb41a24572b3 P 43172513b66542d79f1d327998eb886c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:34:33.397455 24399 raft_consensus.cc:504] T b786f0fbb7c849f1addecb41a24572b3 P 43172513b66542d79f1d327998eb886c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d868a18f455648d9b2d67353e0059e9a" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 40277 } } peers { permanent_uuid: "43172513b66542d79f1d327998eb886c" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36833 } } peers { permanent_uuid: "b04503a3ef6e4d6e833b22c13c65122c" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 43459 } }
I20260808 22:34:33.398569 24399 leader_election.cc:288] T b786f0fbb7c849f1addecb41a24572b3 P 43172513b66542d79f1d327998eb886c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d868a18f455648d9b2d67353e0059e9a (127.31.87.130:40277), b04503a3ef6e4d6e833b22c13c65122c (127.31.87.129:43459)
I20260808 22:34:33.402135 24387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:33.398920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:35697 (local address 127.31.87.129:43459)
0808 22:34:33.399049 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:33.399053 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:33.399173 (+ 120us) server_negotiation.cc:407] Connection header received
0808 22:34:33.399248 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:33.399251 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:33.399297 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:33.399363 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:34:33.399639 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:33.400134 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:33.401207 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:33.401606 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:33.401714 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:33.401768 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:33.401939 (+ 171us) server_negotiation.cc:299] Negotiation successful
0808 22:34:33.402013 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 22:34:33.402135 24370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:33.398839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:43459 (local address 127.31.87.131:35697)
0808 22:34:33.399032 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:33.399047 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:34:33.399156 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:34:33.399383 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:34:33.399386 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:34:33.399403 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:34:33.399524 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:33.399530 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:33.400237 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:33.400240 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:33.401050 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:33.401060 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:33.401717 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:33.401721 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:33.401825 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:33.401843 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 22:34:33.401908 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 22:34:33.401993 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":42}
I20260808 22:34:33.402292 24385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:33.399030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:45021 (local address 127.31.87.130:40277)
0808 22:34:33.399151 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:33.399155 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:33.399363 (+ 208us) server_negotiation.cc:407] Connection header received
0808 22:34:33.399479 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:33.399482 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:33.399529 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:33.399592 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:34:33.399854 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:33.400339 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:33.401389 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:33.401767 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:33.401871 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:33.401919 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:33.402096 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 22:34:33.402163 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 22:34:33.402311 24404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:33.398839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.130:40277 (local address 127.31.87.131:45021)
0808 22:34:33.399270 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:33.399283 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:34:33.399390 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:34:33.399612 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:34:33.399615 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:34:33.399630 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:34:33.399741 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:33.399746 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:33.400441 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:33.400444 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:33.401272 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:33.401281 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:33.401877 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:33.401880 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:33.401988 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:33.402007 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 22:34:33.402071 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 22:34:33.402135 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"mutex_wait_us":51,"thread_start_us":138,"threads_started":1}
I20260808 22:34:33.402926 24085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b786f0fbb7c849f1addecb41a24572b3" candidate_uuid: "43172513b66542d79f1d327998eb886c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d868a18f455648d9b2d67353e0059e9a" is_pre_election: true
I20260808 22:34:33.402926 23882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b786f0fbb7c849f1addecb41a24572b3" candidate_uuid: "43172513b66542d79f1d327998eb886c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b04503a3ef6e4d6e833b22c13c65122c" is_pre_election: true
I20260808 22:34:33.403250 23882 raft_consensus.cc:2454] T b786f0fbb7c849f1addecb41a24572b3 P b04503a3ef6e4d6e833b22c13c65122c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43172513b66542d79f1d327998eb886c in term 0.
I20260808 22:34:33.403250 24085 raft_consensus.cc:2454] T b786f0fbb7c849f1addecb41a24572b3 P d868a18f455648d9b2d67353e0059e9a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43172513b66542d79f1d327998eb886c in term 0.
I20260808 22:34:33.403751 24188 leader_election.cc:302] T b786f0fbb7c849f1addecb41a24572b3 P 43172513b66542d79f1d327998eb886c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43172513b66542d79f1d327998eb886c, d868a18f455648d9b2d67353e0059e9a; no voters:
I20260808 22:34:33.404011 24399 raft_consensus.cc:2781] T b786f0fbb7c849f1addecb41a24572b3 P 43172513b66542d79f1d327998eb886c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:34:33.404093 24399 raft_consensus.cc:482] T b786f0fbb7c849f1addecb41a24572b3 P 43172513b66542d79f1d327998eb886c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:34:33.404142 24399 raft_consensus.cc:3037] T b786f0fbb7c849f1addecb41a24572b3 P 43172513b66542d79f1d327998eb886c [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:34:33.405156 24399 raft_consensus.cc:504] T b786f0fbb7c849f1addecb41a24572b3 P 43172513b66542d79f1d327998eb886c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d868a18f455648d9b2d67353e0059e9a" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 40277 } } peers { permanent_uuid: "43172513b66542d79f1d327998eb886c" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36833 } } peers { permanent_uuid: "b04503a3ef6e4d6e833b22c13c65122c" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 43459 } }
I20260808 22:34:33.405465 24399 leader_election.cc:288] T b786f0fbb7c849f1addecb41a24572b3 P 43172513b66542d79f1d327998eb886c [CANDIDATE]: Term 1 election: Requested vote from peers d868a18f455648d9b2d67353e0059e9a (127.31.87.130:40277), b04503a3ef6e4d6e833b22c13c65122c (127.31.87.129:43459)
I20260808 22:34:33.405836 24085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b786f0fbb7c849f1addecb41a24572b3" candidate_uuid: "43172513b66542d79f1d327998eb886c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d868a18f455648d9b2d67353e0059e9a"
I20260808 22:34:33.405836 23882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b786f0fbb7c849f1addecb41a24572b3" candidate_uuid: "43172513b66542d79f1d327998eb886c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b04503a3ef6e4d6e833b22c13c65122c"
I20260808 22:34:33.405956 24085 raft_consensus.cc:3037] T b786f0fbb7c849f1addecb41a24572b3 P d868a18f455648d9b2d67353e0059e9a [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:34:33.405956 23882 raft_consensus.cc:3037] T b786f0fbb7c849f1addecb41a24572b3 P b04503a3ef6e4d6e833b22c13c65122c [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:34:33.407202 23882 raft_consensus.cc:2454] T b786f0fbb7c849f1addecb41a24572b3 P b04503a3ef6e4d6e833b22c13c65122c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43172513b66542d79f1d327998eb886c in term 1.
I20260808 22:34:33.407202 24085 raft_consensus.cc:2454] T b786f0fbb7c849f1addecb41a24572b3 P d868a18f455648d9b2d67353e0059e9a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43172513b66542d79f1d327998eb886c in term 1.
I20260808 22:34:33.407507 24184 leader_election.cc:302] T b786f0fbb7c849f1addecb41a24572b3 P 43172513b66542d79f1d327998eb886c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 43172513b66542d79f1d327998eb886c, b04503a3ef6e4d6e833b22c13c65122c, d868a18f455648d9b2d67353e0059e9a; no voters:
I20260808 22:34:33.407680 24399 raft_consensus.cc:2781] T b786f0fbb7c849f1addecb41a24572b3 P 43172513b66542d79f1d327998eb886c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:34:33.407896 24399 raft_consensus.cc:686] T b786f0fbb7c849f1addecb41a24572b3 P 43172513b66542d79f1d327998eb886c [term 1 LEADER]: Becoming Leader. State: Replica: 43172513b66542d79f1d327998eb886c, State: Running, Role: LEADER
I20260808 22:34:33.408202 24399 consensus_queue.cc:237] T b786f0fbb7c849f1addecb41a24572b3 P 43172513b66542d79f1d327998eb886c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d868a18f455648d9b2d67353e0059e9a" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 40277 } } peers { permanent_uuid: "43172513b66542d79f1d327998eb886c" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36833 } } peers { permanent_uuid: "b04503a3ef6e4d6e833b22c13c65122c" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 43459 } }
I20260808 22:34:33.411525 23655 catalog_manager.cc:5495] T b786f0fbb7c849f1addecb41a24572b3 P 43172513b66542d79f1d327998eb886c reported cstate change: term changed from 0 to 1, leader changed from <none> to 43172513b66542d79f1d327998eb886c (127.31.87.131). New cstate: current_term: 1 leader_uuid: "43172513b66542d79f1d327998eb886c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d868a18f455648d9b2d67353e0059e9a" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 40277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43172513b66542d79f1d327998eb886c" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36833 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b04503a3ef6e4d6e833b22c13c65122c" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 43459 } health_report { overall_health: UNKNOWN } } }
I20260808 22:34:33.433296 24389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:33.430228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46386 (local address 127.31.87.131:36833)
0808 22:34:33.430350 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:33.430354 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:33.430368 (+ 14us) server_negotiation.cc:407] Connection header received
0808 22:34:33.430469 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:33.430471 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:33.430518 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:33.430604 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:34:33.430779 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:33.431263 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:33.432047 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:33.432267 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:33.432345 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:33.432554 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:34:33.432937 (+ 383us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:34:33.433023 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:33.433079 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 22:34:33.433160 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 22:34:33.438630 23882 raft_consensus.cc:1261] T b786f0fbb7c849f1addecb41a24572b3 P b04503a3ef6e4d6e833b22c13c65122c [term 1 FOLLOWER]: Refusing update from remote peer 43172513b66542d79f1d327998eb886c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:34:33.438630 24085 raft_consensus.cc:1261] T b786f0fbb7c849f1addecb41a24572b3 P d868a18f455648d9b2d67353e0059e9a [term 1 FOLLOWER]: Refusing update from remote peer 43172513b66542d79f1d327998eb886c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:34:33.439287 24405 consensus_queue.cc:1035] T b786f0fbb7c849f1addecb41a24572b3 P 43172513b66542d79f1d327998eb886c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b04503a3ef6e4d6e833b22c13c65122c" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:34:33.439465 24399 consensus_queue.cc:1035] T b786f0fbb7c849f1addecb41a24572b3 P 43172513b66542d79f1d327998eb886c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d868a18f455648d9b2d67353e0059e9a" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 40277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:34:33.447047 24414 mvcc.cc:205] Tried to move back new op lower bound from 7316391827198341120 to 7316391827082870784. Current Snapshot: MvccSnapshot[applied={T|T < 7316391827198341120}]
I20260808 22:34:33.448274 24410 mvcc.cc:205] Tried to move back new op lower bound from 7316391827198341120 to 7316391827082870784. Current Snapshot: MvccSnapshot[applied={T|T < 7316391827198341120 or (T in {7316391827198341120})}]
I20260808 22:34:33.448442 24413 mvcc.cc:205] Tried to move back new op lower bound from 7316391827198341120 to 7316391827082870784. Current Snapshot: MvccSnapshot[applied={T|T < 7316391827198341120}]
I20260808 22:34:33.454526 23655 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:34:33.454665 23655 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:34:33.456789 23655 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8: Sending DeleteTablet for 3 replicas of tablet b786f0fbb7c849f1addecb41a24572b3
I20260808 22:34:33.457540 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 23763
I20260808 22:34:33.457525 23854 tablet_service.cc:1514] Processing DeleteTablet for tablet b786f0fbb7c849f1addecb41a24572b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:34:33 UTC) from {username='kudu', principal='kudu/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:58100
I20260808 22:34:33.457676 24058 tablet_service.cc:1514] Processing DeleteTablet for tablet b786f0fbb7c849f1addecb41a24572b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:34:33 UTC) from {username='kudu', principal='kudu/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:51956
I20260808 22:34:33.458092 24432 tablet_replica.cc:330] T b786f0fbb7c849f1addecb41a24572b3 P d868a18f455648d9b2d67353e0059e9a: stopping tablet replica
I20260808 22:34:33.458155 24265 tablet_service.cc:1514] Processing DeleteTablet for tablet b786f0fbb7c849f1addecb41a24572b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:34:33 UTC) from {username='kudu', principal='kudu/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:46372
I20260808 22:34:33.458376 24432 raft_consensus.cc:2229] T b786f0fbb7c849f1addecb41a24572b3 P d868a18f455648d9b2d67353e0059e9a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:34:33.458568 24433 tablet_replica.cc:330] T b786f0fbb7c849f1addecb41a24572b3 P 43172513b66542d79f1d327998eb886c: stopping tablet replica
I20260808 22:34:33.458832 24433 raft_consensus.cc:2229] T b786f0fbb7c849f1addecb41a24572b3 P 43172513b66542d79f1d327998eb886c [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:34:33.459162 24433 raft_consensus.cc:2258] T b786f0fbb7c849f1addecb41a24572b3 P 43172513b66542d79f1d327998eb886c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:34:33.458686 24432 raft_consensus.cc:2258] T b786f0fbb7c849f1addecb41a24572b3 P d868a18f455648d9b2d67353e0059e9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:34:33.462983 24433 ts_tablet_manager.cc:1905] T b786f0fbb7c849f1addecb41a24572b3 P 43172513b66542d79f1d327998eb886c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:34:33.465225 24432 ts_tablet_manager.cc:1905] T b786f0fbb7c849f1addecb41a24572b3 P d868a18f455648d9b2d67353e0059e9a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:34:33.465780 24433 ts_tablet_manager.cc:1918] T b786f0fbb7c849f1addecb41a24572b3 P 43172513b66542d79f1d327998eb886c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:34:33.465862 24433 log.cc:1196] T b786f0fbb7c849f1addecb41a24572b3 P 43172513b66542d79f1d327998eb886c: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-2/wal/wals/b786f0fbb7c849f1addecb41a24572b3
I20260808 22:34:33.466172 24433 ts_tablet_manager.cc:1939] T b786f0fbb7c849f1addecb41a24572b3 P 43172513b66542d79f1d327998eb886c: Deleting consensus metadata
I20260808 22:34:33.467262 23635 catalog_manager.cc:4841] TS 43172513b66542d79f1d327998eb886c (127.31.87.131:36833): tablet b786f0fbb7c849f1addecb41a24572b3 (table test-table [id=a4e97a00f7f140fd99b5139d6ac80816]) successfully deleted
I20260808 22:34:33.468349 24432 ts_tablet_manager.cc:1918] T b786f0fbb7c849f1addecb41a24572b3 P d868a18f455648d9b2d67353e0059e9a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:34:33.468454 24432 log.cc:1196] T b786f0fbb7c849f1addecb41a24572b3 P d868a18f455648d9b2d67353e0059e9a: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-1/wal/wals/b786f0fbb7c849f1addecb41a24572b3
I20260808 22:34:33.468824 24432 ts_tablet_manager.cc:1939] T b786f0fbb7c849f1addecb41a24572b3 P d868a18f455648d9b2d67353e0059e9a: Deleting consensus metadata
I20260808 22:34:33.471037 23635 catalog_manager.cc:4841] TS d868a18f455648d9b2d67353e0059e9a (127.31.87.130:40277): tablet b786f0fbb7c849f1addecb41a24572b3 (table test-table [id=a4e97a00f7f140fd99b5139d6ac80816]) successfully deleted
W20260808 22:34:33.480006 23632 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.87.129:43459 (error 108)
I20260808 22:34:33.480274 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 23956
I20260808 22:34:33.482339 24386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:33.480807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:43459 (local address 127.0.0.1:58112)
0808 22:34:33.482180 (+ 1373us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:33.482252 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.129:43459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1286}
W20260808 22:34:33.483618 23632 catalog_manager.cc:4584] TS b04503a3ef6e4d6e833b22c13c65122c (127.31.87.129:43459): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b786f0fbb7c849f1addecb41a24572b3: Network error: Client connection negotiation failed: client connection to 127.31.87.129:43459: connect: Connection refused (error 111)
I20260808 22:34:33.489676 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 24163
I20260808 22:34:33.504383 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 23604
I20260808 22:34:53.516640 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:37545
--webserver_interface=127.31.87.190
--webserver_port=43477
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:34763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:37545
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:34:53.626111 24435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:34:53.626333 24435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:34:53.626374 24435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:34:53.630254 24435 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 22:34:53.630298 24435 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 22:34:53.630312 24435 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 22:34:53.630337 24435 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:34:53.630368 24435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:34:53.630396 24435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:34:53.630424 24435 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:34:53.630453 24435 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:34:53.635504 24435 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:34763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:37545
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:37545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.87.190
--webserver_port=43477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24435
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:34:53.636421 24435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:34:53.637390 24435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:34:53.650161 24445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:34:53.650228 24435 server_base.cc:1034] running on GCE node
W20260808 22:34:53.650197 24444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:34:53.650180 24448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:34:53.650873 24435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:34:53.651921 24435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:34:53.653071 24435 hybrid_clock.cc:648] HybridClock initialized: now 1786228493653052 us; error 33 us; skew 500 ppm
Aug 08 22:34:53 dist-test-slave-266d krb5kdc[23581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228493, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:34:53.655660 24435 init.cc:377] Logged in from keytab as kudu/127.31.87.190@KRBTEST.COM (short username kudu)
I20260808 22:34:53.656488 24435 webserver.cc:460] Webserver started at http://127.31.87.190:43477/ using document root <none> and password file <none>
I20260808 22:34:53.657009 24435 fs_manager.cc:359] Metadata directory not provided
I20260808 22:34:53.657083 24435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:34:53.660614 24435 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260808 22:34:53.662631 24457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:53.663724 24435 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260808 22:34:53.663833 24435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "5c4f0c26263f4a79994b0c236d72fad8"
format_stamp: "Formatted at 2026-08-08 22:34:32 on dist-test-slave-266d"
server_key: "f44ab1ebc0fd14d0def43d3ad5410ed0"
server_key_iv: "f97f82f6979387d4b1fdfa875ec3a870"
server_key_version: "encryptionkey@0"
I20260808 22:34:53.664376 24435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:34:53.680620 24435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:34:53.683641 24435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:34:53.683830 24435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:34:53.690727 24435 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.190:37545
I20260808 22:34:53.690966 24542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.190:37545 every 8 connection(s)
I20260808 22:34:53.691807 24435 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:34:53.693195 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 24435
I20260808 22:34:53.693596 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.129:43459
--local_ip_for_outbound_sockets=127.31.87.129
--tserver_master_addrs=127.31.87.190:37545
--webserver_port=45475
--webserver_interface=127.31.87.129
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab
--principal=kudu/127.31.87.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:34763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 22:34:53.696532 24543 sys_catalog.cc:263] Verifying existing consensus state
I20260808 22:34:53.698252 24543 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8: Bootstrap starting.
I20260808 22:34:53.715018 24543 log.cc:824] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8: Log is configured to *not* fsync() on all Append() calls
I20260808 22:34:53.724188 24543 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 22:34:53.724527 24543 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8: Bootstrap complete.
I20260808 22:34:53.728375 24543 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c4f0c26263f4a79994b0c236d72fad8" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 37545 } }
I20260808 22:34:53.728765 24543 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c4f0c26263f4a79994b0c236d72fad8, State: Initialized, Role: FOLLOWER
I20260808 22:34:53.729283 24543 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c4f0c26263f4a79994b0c236d72fad8" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 37545 } }
I20260808 22:34:53.729419 24543 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:34:53.729499 24543 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:34:53.729614 24543 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8 [term 1 FOLLOWER]: Advancing to term 2
I20260808 22:34:53.731406 24543 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c4f0c26263f4a79994b0c236d72fad8" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 37545 } }
I20260808 22:34:53.731760 24543 leader_election.cc:302] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c4f0c26263f4a79994b0c236d72fad8; no voters:
I20260808 22:34:53.732066 24543 leader_election.cc:288] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 22:34:53.732172 24550 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 22:34:53.732383 24550 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8 [term 2 LEADER]: Becoming Leader. State: Replica: 5c4f0c26263f4a79994b0c236d72fad8, State: Running, Role: LEADER
I20260808 22:34:53.732745 24550 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c4f0c26263f4a79994b0c236d72fad8" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 37545 } }
I20260808 22:34:53.733321 24543 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:34:53.734138 24551 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "5c4f0c26263f4a79994b0c236d72fad8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c4f0c26263f4a79994b0c236d72fad8" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 37545 } } }
I20260808 22:34:53.734272 24551 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8 [sys.catalog]: This master's current role is: LEADER
I20260808 22:34:53.734916 24555 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c4f0c26263f4a79994b0c236d72fad8. Latest consensus state: current_term: 2 leader_uuid: "5c4f0c26263f4a79994b0c236d72fad8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c4f0c26263f4a79994b0c236d72fad8" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 37545 } } }
I20260808 22:34:53.735141 24555 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8 [sys.catalog]: This master's current role is: LEADER
I20260808 22:34:53.734925 24564 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:34:53.740051 24564 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:34:53.740649 24564 catalog_manager.cc:1259] Loaded cluster ID: d66d2e3ed939478ba6705e1ab06307ff
I20260808 22:34:53.740708 24564 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:34:53.743103 24564 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:34:53.744397 24564 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8: Loaded TSK: 0
I20260808 22:34:53.762336 24564 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8: Generated new TSK 1
I20260808 22:34:53.762563 24564 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 22:34:53.810482 24548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:34:53.810705 24548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:34:53.810745 24548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:34:53.814611 24548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:34:53.814663 24548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:34:53.814720 24548 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.129
I20260808 22:34:53.820056 24548 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:34763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab
--principal=kudu/127.31.87.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.129:43459
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.87.129
--webserver_port=45475
--tserver_master_addrs=127.31.87.190:37545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24548
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.129
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:34:53.821055 24548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:34:53.822057 24548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:34:53.832221 24589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:34:53.832243 24593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:34:53.832440 24548 server_base.cc:1034] running on GCE node
W20260808 22:34:53.832242 24588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:34:53.832989 24548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:34:53.833848 24548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:34:53.835047 24548 hybrid_clock.cc:648] HybridClock initialized: now 1786228493835020 us; error 35 us; skew 500 ppm
Aug 08 22:34:53 dist-test-slave-266d krb5kdc[23581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228493, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:34:53.838565 24548 init.cc:377] Logged in from keytab as kudu/127.31.87.129@KRBTEST.COM (short username kudu)
I20260808 22:34:53.839603 24548 webserver.cc:460] Webserver started at http://127.31.87.129:45475/ using document root <none> and password file <none>
I20260808 22:34:53.840353 24548 fs_manager.cc:359] Metadata directory not provided
I20260808 22:34:53.840454 24548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:34:53.845498 24548 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 22:34:53.848527 24606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:53.849834 24548 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 22:34:53.849928 24548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-0/wal
uuid: "b04503a3ef6e4d6e833b22c13c65122c"
format_stamp: "Formatted at 2026-08-08 22:34:32 on dist-test-slave-266d"
server_key: "d9262e12f89f8cf4cb3863fb25fd7135"
server_key_iv: "7fb3e8b41f829b8c20ae8a72976eefe1"
server_key_version: "encryptionkey@0"
I20260808 22:34:53.850420 24548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:34:53.867168 24548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:34:53.870147 24548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:34:53.870317 24548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:34:53.870889 24548 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:34:53.871959 24615 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 22:34:53.875830 24548 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 22:34:53.875881 24548 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260808 22:34:53.875924 24548 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 22:34:53.877573 24548 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 22:34:53.877629 24548 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260808 22:34:53.877899 24615 tablet_bootstrap.cc:492] T b786f0fbb7c849f1addecb41a24572b3 P b04503a3ef6e4d6e833b22c13c65122c: Bootstrap starting.
I20260808 22:34:53.887895 24548 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.129:43459
I20260808 22:34:53.887965 24790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.129:43459 every 8 connection(s)
I20260808 22:34:53.889063 24548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
I20260808 22:34:53.891048 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 24548
I20260808 22:34:53.891538 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.130:40277
--local_ip_for_outbound_sockets=127.31.87.130
--tserver_master_addrs=127.31.87.190:37545
--webserver_port=45801
--webserver_interface=127.31.87.130
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab
--principal=kudu/127.31.87.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:34763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 22:34:53.898384 24615 log.cc:824] T b786f0fbb7c849f1addecb41a24572b3 P b04503a3ef6e4d6e833b22c13c65122c: Log is configured to *not* fsync() on all Append() calls
Aug 08 22:34:53 dist-test-slave-266d krb5kdc[23581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228493, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.129@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:34:53.906594 24615 tablet_bootstrap.cc:492] T b786f0fbb7c849f1addecb41a24572b3 P b04503a3ef6e4d6e833b22c13c65122c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 22:34:53.907024 24615 tablet_bootstrap.cc:492] T b786f0fbb7c849f1addecb41a24572b3 P b04503a3ef6e4d6e833b22c13c65122c: Bootstrap complete.
I20260808 22:34:53.908294 24615 ts_tablet_manager.cc:1397] T b786f0fbb7c849f1addecb41a24572b3 P b04503a3ef6e4d6e833b22c13c65122c: Time spent bootstrapping tablet: real 0.031s user 0.010s sys 0.017s
I20260808 22:34:53.908547 24798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:53.891124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.129:42749)
0808 22:34:53.891678 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:53.891740 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 22:34:53.892727 (+ 987us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:34:53.895153 (+ 2426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:34:53.895162 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:34:53.895643 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:34:53.896382 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:53.896404 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:53.897812 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:53.897817 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:53.898739 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:53.898752 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:53.899776 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:53.899781 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:53.899976 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:53.900668 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:34:53.900691 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:34:53.902587 (+ 1896us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:34:53.905161 (+ 2574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:53.905168 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:53.905182 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:53.905533 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:53.905948 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:53.905952 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:53.905954 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:53.906134 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:53.906872 (+ 738us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:34:53.906878 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:34:53.907150 (+ 272us) client_negotiation.cc:769] Sending connection context
0808 22:34:53.907346 (+ 196us) client_negotiation.cc:240] Negotiation successful
0808 22:34:53.907650 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":131,"threads_started":1}
I20260808 22:34:53.908644 24801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:53.893470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:42749 (local address 127.31.87.190:37545)
0808 22:34:53.893885 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:53.893906 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:53.893955 (+ 49us) server_negotiation.cc:407] Connection header received
0808 22:34:53.894242 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:53.894258 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:53.894680 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:53.895069 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:34:53.896578 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:53.897644 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:53.898872 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:53.899617 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:53.899792 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:53.902808 (+ 3016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:34:53.902815 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:34:53.902830 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:34:53.902866 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:34:53.905011 (+ 2145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:53.905659 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:53.905665 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:53.905671 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:53.905826 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:53.906244 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:53.906248 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:53.906250 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:53.906734 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:34:53.906876 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:53.907492 (+ 616us) server_negotiation.cc:299] Negotiation successful
0808 22:34:53.907792 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":121,"threads_started":1}
I20260808 22:34:53.911479 24795 heartbeater.cc:347] Connected to a master server at 127.31.87.190:37545
I20260808 22:34:53.911101 24615 raft_consensus.cc:348] T b786f0fbb7c849f1addecb41a24572b3 P b04503a3ef6e4d6e833b22c13c65122c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d868a18f455648d9b2d67353e0059e9a" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 40277 } } peers { permanent_uuid: "43172513b66542d79f1d327998eb886c" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36833 } } peers { permanent_uuid: "b04503a3ef6e4d6e833b22c13c65122c" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 43459 } }
I20260808 22:34:53.911610 24615 raft_consensus.cc:729] T b786f0fbb7c849f1addecb41a24572b3 P b04503a3ef6e4d6e833b22c13c65122c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b04503a3ef6e4d6e833b22c13c65122c, State: Initialized, Role: FOLLOWER
I20260808 22:34:53.911707 24795 heartbeater.cc:464] Registering TS with master...
I20260808 22:34:53.912123 24615 consensus_queue.cc:260] T b786f0fbb7c849f1addecb41a24572b3 P b04503a3ef6e4d6e833b22c13c65122c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d868a18f455648d9b2d67353e0059e9a" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 40277 } } peers { permanent_uuid: "43172513b66542d79f1d327998eb886c" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36833 } } peers { permanent_uuid: "b04503a3ef6e4d6e833b22c13c65122c" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 43459 } }
I20260808 22:34:53.912209 24795 heartbeater.cc:511] Master 127.31.87.190:37545 requested a full tablet report, sending...
I20260808 22:34:53.913009 24615 ts_tablet_manager.cc:1428] T b786f0fbb7c849f1addecb41a24572b3 P b04503a3ef6e4d6e833b22c13c65122c: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260808 22:34:53.914707 24488 ts_manager.cc:194] Registered new tserver with Master: b04503a3ef6e4d6e833b22c13c65122c (127.31.87.129:43459)
I20260808 22:34:53.918021 24488 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.129@KRBTEST.COM'} at 127.31.87.129:42749
I20260808 22:34:53.920629 24795 heartbeater.cc:503] Master 127.31.87.190:37545 was elected leader, sending a full tablet report...
Aug 08 22:34:53 dist-test-slave-266d krb5kdc[23581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228493, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for kudu/127.31.87.129@KRBTEST.COM
I20260808 22:34:53.928082 24805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:53.917410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:43459 (local address 127.0.0.1:45868)
0808 22:34:53.917961 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:53.918006 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 22:34:53.918120 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:34:53.918602 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:34:53.918607 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:34:53.919857 (+ 1250us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:34:53.919968 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:53.919976 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:53.921127 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:53.921131 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:53.921799 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:53.921810 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:53.922479 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:53.922483 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:53.922605 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:53.923100 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:34:53.923112 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:34:53.924582 (+ 1470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:34:53.926314 (+ 1732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:53.926324 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:53.926330 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:53.926588 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:53.926886 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:53.926890 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:53.926892 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:53.926981 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:53.927437 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:34:53.927443 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:34:53.927572 (+ 129us) client_negotiation.cc:769] Sending connection context
0808 22:34:53.927701 (+ 129us) client_negotiation.cc:240] Negotiation successful
0808 22:34:53.927893 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":119,"threads_started":1}
I20260808 22:34:53.928117 24806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:53.917487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45868 (local address 127.31.87.129:43459)
0808 22:34:53.918154 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:53.918169 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:53.918193 (+ 24us) server_negotiation.cc:407] Connection header received
0808 22:34:53.918262 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:53.918268 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:53.918456 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:53.918603 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:34:53.920095 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:53.920960 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:53.921931 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:53.922357 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:53.922464 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:53.924722 (+ 2258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:34:53.924731 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:34:53.924741 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:34:53.924763 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:34:53.926203 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:53.926703 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:53.926708 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:53.926709 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:53.926794 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:53.927076 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:53.927080 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:53.927082 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:53.927318 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:34:53.927446 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:53.927796 (+ 350us) server_negotiation.cc:299] Negotiation successful
0808 22:34:53.927950 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":104,"threads_started":1}
I20260808 22:34:53.929204 24681 tablet_service.cc:1514] Processing DeleteTablet for tablet b786f0fbb7c849f1addecb41a24572b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:34:33 UTC) from {username='kudu', principal='kudu/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:45868
I20260808 22:34:53.929970 24807 tablet_replica.cc:330] stopping tablet replica
I20260808 22:34:53.930238 24807 raft_consensus.cc:2229] T b786f0fbb7c849f1addecb41a24572b3 P b04503a3ef6e4d6e833b22c13c65122c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:34:53.930452 24807 raft_consensus.cc:2258] T b786f0fbb7c849f1addecb41a24572b3 P b04503a3ef6e4d6e833b22c13c65122c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:34:53.931365 24807 ts_tablet_manager.cc:1905] T b786f0fbb7c849f1addecb41a24572b3 P b04503a3ef6e4d6e833b22c13c65122c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:34:53.934872 24807 ts_tablet_manager.cc:1918] T b786f0fbb7c849f1addecb41a24572b3 P b04503a3ef6e4d6e833b22c13c65122c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:34:53.934958 24807 log.cc:1196] T b786f0fbb7c849f1addecb41a24572b3 P b04503a3ef6e4d6e833b22c13c65122c: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-0/wal/wals/b786f0fbb7c849f1addecb41a24572b3
I20260808 22:34:53.935253 24807 ts_tablet_manager.cc:1939] T b786f0fbb7c849f1addecb41a24572b3 P b04503a3ef6e4d6e833b22c13c65122c: Deleting consensus metadata
I20260808 22:34:53.936905 24458 catalog_manager.cc:4841] TS b04503a3ef6e4d6e833b22c13c65122c (127.31.87.129:43459): tablet b786f0fbb7c849f1addecb41a24572b3 (table test-table [id=a4e97a00f7f140fd99b5139d6ac80816]) successfully deleted
W20260808 22:34:54.007447 24799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:34:54.007676 24799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:34:54.007730 24799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:34:54.011652 24799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:34:54.011701 24799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:34:54.011782 24799 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.130
I20260808 22:34:54.016726 24799 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:34763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab
--principal=kudu/127.31.87.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.130:40277
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.87.130
--webserver_port=45801
--tserver_master_addrs=127.31.87.190:37545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.130
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:34:54.017648 24799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:34:54.018558 24799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:34:54.027779 24813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:34:54.027827 24799 server_base.cc:1034] running on GCE node
W20260808 22:34:54.027787 24817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:34:54.028034 24814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:34:54.028421 24799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:34:54.029002 24799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:34:54.030159 24799 hybrid_clock.cc:648] HybridClock initialized: now 1786228494030136 us; error 35 us; skew 500 ppm
Aug 08 22:34:54 dist-test-slave-266d krb5kdc[23581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228494, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:34:54.032670 24799 init.cc:377] Logged in from keytab as kudu/127.31.87.130@KRBTEST.COM (short username kudu)
I20260808 22:34:54.033573 24799 webserver.cc:460] Webserver started at http://127.31.87.130:45801/ using document root <none> and password file <none>
I20260808 22:34:54.034142 24799 fs_manager.cc:359] Metadata directory not provided
I20260808 22:34:54.034224 24799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:34:54.037979 24799 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.004s
I20260808 22:34:54.039898 24827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:54.041019 24799 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:34:54.041146 24799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-1/wal
uuid: "d868a18f455648d9b2d67353e0059e9a"
format_stamp: "Formatted at 2026-08-08 22:34:32 on dist-test-slave-266d"
server_key: "0a91ad264b934d578ca5f97b838f95fc"
server_key_iv: "a30caef0655e0ace7cde4c2a6f13e92a"
server_key_version: "encryptionkey@0"
I20260808 22:34:54.041672 24799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:34:54.057372 24799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:34:54.064046 24799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:34:54.064216 24799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:34:54.064739 24799 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:34:54.065557 24799 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:34:54.065604 24799 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:54.065654 24799 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:34:54.065682 24799 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:54.074069 24799 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.130:40277
I20260808 22:34:54.074093 25013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.130:40277 every 8 connection(s)
I20260808 22:34:54.074981 24799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
I20260808 22:34:54.082002 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 24799
I20260808 22:34:54.082479 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.131:36833
--local_ip_for_outbound_sockets=127.31.87.131
--tserver_master_addrs=127.31.87.190:37545
--webserver_port=35617
--webserver_interface=127.31.87.131
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab
--principal=kudu/127.31.87.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:34763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 22:34:54 dist-test-slave-266d krb5kdc[23581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228494, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.130@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:34:54.087602 24801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:54.076644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:52711 (local address 127.31.87.190:37545)
0808 22:34:54.076776 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:54.076780 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:54.077784 (+ 1004us) server_negotiation.cc:407] Connection header received
0808 22:34:54.078562 (+ 778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:54.078565 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:54.078615 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:54.078686 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:34:54.079731 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:54.080265 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:54.081343 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:54.081573 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:54.081651 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:54.084204 (+ 2553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:34:54.084209 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:34:54.084211 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:34:54.084243 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:34:54.085757 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:54.086203 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:54.086207 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:54.086209 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:54.086297 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:54.086569 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:54.086572 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:54.086574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:54.086781 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:34:54.086876 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:54.087297 (+ 421us) server_negotiation.cc:299] Negotiation successful
0808 22:34:54.087429 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 22:34:54.088238 25016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:54.077011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.130:52711)
0808 22:34:54.077646 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:54.077683 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 22:34:54.078402 (+ 719us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:34:54.078844 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:34:54.078851 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:34:54.079178 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:34:54.079616 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:54.079629 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:54.080361 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:54.080363 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:54.081249 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:54.081258 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:54.081682 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:54.081684 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:54.081895 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:54.082446 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:34:54.082465 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:34:54.084019 (+ 1554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:34:54.085863 (+ 1844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:54.085868 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:54.085878 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:54.086107 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:54.086378 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:54.086380 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:54.086381 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:54.086492 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:54.086869 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:34:54.086873 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:34:54.087072 (+ 199us) client_negotiation.cc:769] Sending connection context
0808 22:34:54.087216 (+ 144us) client_negotiation.cc:240] Negotiation successful
0808 22:34:54.087465 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":119,"threads_started":1}
I20260808 22:34:54.089320 25014 heartbeater.cc:347] Connected to a master server at 127.31.87.190:37545
I20260808 22:34:54.089572 25014 heartbeater.cc:464] Registering TS with master...
I20260808 22:34:54.090127 25014 heartbeater.cc:511] Master 127.31.87.190:37545 requested a full tablet report, sending...
I20260808 22:34:54.091221 24488 ts_manager.cc:194] Registered new tserver with Master: d868a18f455648d9b2d67353e0059e9a (127.31.87.130:40277)
I20260808 22:34:54.091945 24488 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.130@KRBTEST.COM'} at 127.31.87.130:52711
W20260808 22:34:54.193195 25017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:34:54.193420 25017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:34:54.193459 25017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:34:54.197284 25017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:34:54.197335 25017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:34:54.197388 25017 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.131
I20260808 22:34:54.202325 25017 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:34763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab
--principal=kudu/127.31.87.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.131:36833
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.87.131
--webserver_port=35617
--tserver_master_addrs=127.31.87.190:37545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25017
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.131
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:34:54.203202 25017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:34:54.204098 25017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:34:54.213032 25025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:34:54.213111 25035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:34:54.213193 25017 server_base.cc:1034] running on GCE node
W20260808 22:34:54.213078 25027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:34:54.213680 25017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:34:54.214260 25017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:34:54.215438 25017 hybrid_clock.cc:648] HybridClock initialized: now 1786228494215419 us; error 26 us; skew 500 ppm
Aug 08 22:34:54 dist-test-slave-266d krb5kdc[23581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228494, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:34:54.217984 25017 init.cc:377] Logged in from keytab as kudu/127.31.87.131@KRBTEST.COM (short username kudu)
I20260808 22:34:54.218818 25017 webserver.cc:460] Webserver started at http://127.31.87.131:35617/ using document root <none> and password file <none>
I20260808 22:34:54.219342 25017 fs_manager.cc:359] Metadata directory not provided
I20260808 22:34:54.219424 25017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:34:54.223215 25017 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 22:34:54.225390 25048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:54.226550 25017 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 22:34:54.226655 25017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-2/wal
uuid: "43172513b66542d79f1d327998eb886c"
format_stamp: "Formatted at 2026-08-08 22:34:33 on dist-test-slave-266d"
server_key: "b28fe866acfd0353c226c51e0002da45"
server_key_iv: "13c50f9463f0a73c1d1ed778c9b16ed4"
server_key_version: "encryptionkey@0"
I20260808 22:34:54.227164 25017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:34:54.239243 25017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:34:54.241894 25017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:34:54.242053 25017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:34:54.242575 25017 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:34:54.243345 25017 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:34:54.243391 25017 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:54.243443 25017 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:34:54.243469 25017 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:34:54.252099 25017 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.131:36833
I20260808 22:34:54.252127 25222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.131:36833 every 8 connection(s)
I20260808 22:34:54.253002 25017 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
I20260808 22:34:54.259454 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 25017
Aug 08 22:34:54 dist-test-slave-266d krb5kdc[23581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228494, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.131@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:34:54.265964 24801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:54.254764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:51403 (local address 127.31.87.190:37545)
0808 22:34:54.254893 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:54.254898 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:54.255890 (+ 992us) server_negotiation.cc:407] Connection header received
0808 22:34:54.256703 (+ 813us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:54.256705 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:54.256753 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:54.256824 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:34:54.257941 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:54.258449 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:54.259514 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:54.259809 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:54.259910 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:54.262461 (+ 2551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:34:54.262465 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:34:54.262468 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:34:54.262494 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:34:54.263973 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:54.264460 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:54.264464 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:54.264465 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:54.264543 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:54.264888 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:54.264890 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:54.264891 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:54.265064 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:34:54.265177 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:54.265720 (+ 543us) server_negotiation.cc:299] Negotiation successful
0808 22:34:54.265845 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 22:34:54.266984 25230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:54.255145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.131:51403)
0808 22:34:54.255744 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:54.255790 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 22:34:54.256528 (+ 738us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:34:54.256963 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:34:54.256969 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:34:54.257361 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:34:54.257815 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:54.257828 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:54.258533 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:54.258535 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:54.259400 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:54.259409 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:54.259974 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:54.259978 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:54.260189 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:54.260753 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:34:54.260769 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:34:54.262308 (+ 1539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:34:54.264078 (+ 1770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:54.264084 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:54.264098 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:54.264364 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:54.264628 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:34:54.264632 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:34:54.264634 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:34:54.264789 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:34:54.265176 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:34:54.265182 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:34:54.265475 (+ 293us) client_negotiation.cc:769] Sending connection context
0808 22:34:54.265690 (+ 215us) client_negotiation.cc:240] Negotiation successful
0808 22:34:54.266000 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":106,"threads_started":1}
I20260808 22:34:54.268111 25235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:54.260094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33690 (local address 127.31.87.190:37545)
0808 22:34:54.260406 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:54.260410 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:54.260441 (+ 31us) server_negotiation.cc:407] Connection header received
0808 22:34:54.260573 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:54.260576 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:54.260617 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:54.260687 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:34:54.261425 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:54.261929 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:54.262776 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:54.263076 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:54.263184 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:54.264356 (+ 1172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:34:54.264359 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:34:54.264362 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:34:54.264386 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:34:54.265949 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:54.266498 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:54.266501 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:54.266503 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:54.266583 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:54.266854 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:54.266857 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:54.266858 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:54.267062 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:34:54.267839 (+ 777us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:54.267907 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 22:34:54.267997 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":92,"threads_started":1}
I20260808 22:34:54.268146 25227 heartbeater.cc:347] Connected to a master server at 127.31.87.190:37545
I20260808 22:34:54.268376 25227 heartbeater.cc:464] Registering TS with master...
I20260808 22:34:54.268914 25227 heartbeater.cc:511] Master 127.31.87.190:37545 requested a full tablet report, sending...
I20260808 22:34:54.270005 24488 ts_manager.cc:194] Registered new tserver with Master: 43172513b66542d79f1d327998eb886c (127.31.87.131:36833)
I20260808 22:34:54.270512 24488 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.131@KRBTEST.COM'} at 127.31.87.131:51403
I20260808 22:34:54.281279 32094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:34:54.284557 25235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:54.281804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33696 (local address 127.31.87.190:37545)
0808 22:34:54.281902 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:54.281905 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:54.282062 (+ 157us) server_negotiation.cc:407] Connection header received
0808 22:34:54.282179 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:54.282181 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:54.282264 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:54.282340 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:34:54.282502 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:54.282973 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:54.283707 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:54.283909 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:54.283975 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:54.284186 (+ 211us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:34:54.284298 (+ 112us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0808 22:34:54.284475 (+ 177us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33696: token expired
Metrics: {"server-negotiator.queue_time_us":36}
W20260808 22:34:54.284605 25235 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33696: token expired
W20260808 22:34:54.284925 24375 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.31.87.190:37545)
I20260808 22:34:54.290812 25235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:54.285237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33698 (local address 127.31.87.190:37545)
0808 22:34:54.285347 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:54.285350 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:54.285361 (+ 11us) server_negotiation.cc:407] Connection header received
0808 22:34:54.285429 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:54.285431 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:54.285467 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:54.285523 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:34:54.286013 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:54.286485 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:54.287153 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:54.287346 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:54.287416 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:54.288509 (+ 1093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:34:54.288512 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:34:54.288513 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:34:54.288528 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:34:54.289702 (+ 1174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:54.290023 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:54.290026 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:54.290027 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:54.290087 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:34:54.290258 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:34:54.290260 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:34:54.290261 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:34:54.290395 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:34:54.290481 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:54.290636 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 22:34:54.290718 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 22:34:54.292786 24488 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33698:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:34:54.295810 24488 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:34:54.302829 24681 tablet_service.cc:1467] Processing CreateTablet for tablet d2f7ff0814894106a1c3da6a1723ef2c (DEFAULT_TABLE table=test-table [id=113bbc9d1c944860823b4eaeddfbb852]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:34:54.303476 24681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2f7ff0814894106a1c3da6a1723ef2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:34:54.306023 24615 tablet_bootstrap.cc:492] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c: Bootstrap starting.
I20260808 22:34:54.306866 24615 tablet_bootstrap.cc:654] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c: Neither blocks nor log segments found. Creating new log.
I20260808 22:34:54.307538 25241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:54.302073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.131:36833 (local address 127.0.0.1:42792)
0808 22:34:54.302368 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:54.302380 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:34:54.302457 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:34:54.303257 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:34:54.303260 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:34:54.303287 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:34:54.303367 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:54.303373 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:54.304589 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:54.304593 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:54.305902 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:54.305913 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:54.307065 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:54.307069 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:54.307185 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:54.307240 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 22:34:54.307325 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 22:34:54.307399 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":53,"thread_start_us":60,"threads_started":1}
I20260808 22:34:54.307605 25243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:54.302482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42792 (local address 127.31.87.131:36833)
0808 22:34:54.302826 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:54.302837 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:54.302860 (+ 23us) server_negotiation.cc:407] Connection header received
0808 22:34:54.302931 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:54.302936 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:54.303117 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:54.303272 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:34:54.303459 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:54.304450 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:54.306040 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:54.306956 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:54.307056 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:54.307192 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:54.307369 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 22:34:54.307463 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":140,"threads_started":1}
I20260808 22:34:54.307893 24805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:54.301877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.130:40277 (local address 127.0.0.1:58664)
0808 22:34:54.301976 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:54.301990 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:34:54.302063 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:34:54.302899 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:34:54.302902 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:34:54.302919 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:34:54.303009 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:54.303015 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:54.304477 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:54.304481 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:54.305902 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:54.305913 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:54.307531 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:54.307534 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:54.307630 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:54.307655 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 22:34:54.307705 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 22:34:54.307781 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29,"mutex_wait_us":85}
I20260808 22:34:54.308331 24615 tablet_bootstrap.cc:492] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c: No bootstrap required, opened a new log
I20260808 22:34:54.308416 24615 ts_tablet_manager.cc:1397] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 22:34:54.308465 25242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:54.302049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58664 (local address 127.31.87.130:40277)
0808 22:34:54.302409 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:54.302422 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:54.302444 (+ 22us) server_negotiation.cc:407] Connection header received
0808 22:34:54.302522 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:54.302528 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:54.302750 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:54.302904 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:34:54.303141 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:54.304377 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:54.306040 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:54.307401 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:54.308061 (+ 660us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:54.308192 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:54.308238 (+ 46us) server_negotiation.cc:299] Negotiation successful
0808 22:34:54.308333 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":161,"threads_started":1}
I20260808 22:34:54.308851 24615 raft_consensus.cc:348] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d868a18f455648d9b2d67353e0059e9a" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 40277 } } peers { permanent_uuid: "43172513b66542d79f1d327998eb886c" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36833 } } peers { permanent_uuid: "b04503a3ef6e4d6e833b22c13c65122c" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 43459 } }
I20260808 22:34:54.308957 24615 raft_consensus.cc:374] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:34:54.308991 24615 raft_consensus.cc:729] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b04503a3ef6e4d6e833b22c13c65122c, State: Initialized, Role: FOLLOWER
I20260808 22:34:54.309137 24615 consensus_queue.cc:260] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d868a18f455648d9b2d67353e0059e9a" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 40277 } } peers { permanent_uuid: "43172513b66542d79f1d327998eb886c" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36833 } } peers { permanent_uuid: "b04503a3ef6e4d6e833b22c13c65122c" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 43459 } }
I20260808 22:34:54.309401 24615 ts_tablet_manager.cc:1428] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 22:34:54.310570 25127 tablet_service.cc:1467] Processing CreateTablet for tablet d2f7ff0814894106a1c3da6a1723ef2c (DEFAULT_TABLE table=test-table [id=113bbc9d1c944860823b4eaeddfbb852]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:34:54.311034 24909 tablet_service.cc:1467] Processing CreateTablet for tablet d2f7ff0814894106a1c3da6a1723ef2c (DEFAULT_TABLE table=test-table [id=113bbc9d1c944860823b4eaeddfbb852]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:34:54.311471 25127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2f7ff0814894106a1c3da6a1723ef2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:34:54.311964 24909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2f7ff0814894106a1c3da6a1723ef2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:34:54.317866 25245 tablet_bootstrap.cc:492] T d2f7ff0814894106a1c3da6a1723ef2c P 43172513b66542d79f1d327998eb886c: Bootstrap starting.
I20260808 22:34:54.317865 25247 tablet_bootstrap.cc:492] T d2f7ff0814894106a1c3da6a1723ef2c P d868a18f455648d9b2d67353e0059e9a: Bootstrap starting.
I20260808 22:34:54.320080 25245 tablet_bootstrap.cc:654] T d2f7ff0814894106a1c3da6a1723ef2c P 43172513b66542d79f1d327998eb886c: Neither blocks nor log segments found. Creating new log.
I20260808 22:34:54.320115 25247 tablet_bootstrap.cc:654] T d2f7ff0814894106a1c3da6a1723ef2c P d868a18f455648d9b2d67353e0059e9a: Neither blocks nor log segments found. Creating new log.
I20260808 22:34:54.320894 25245 log.cc:824] T d2f7ff0814894106a1c3da6a1723ef2c P 43172513b66542d79f1d327998eb886c: Log is configured to *not* fsync() on all Append() calls
I20260808 22:34:54.320904 25247 log.cc:824] T d2f7ff0814894106a1c3da6a1723ef2c P d868a18f455648d9b2d67353e0059e9a: Log is configured to *not* fsync() on all Append() calls
I20260808 22:34:54.322513 25247 tablet_bootstrap.cc:492] T d2f7ff0814894106a1c3da6a1723ef2c P d868a18f455648d9b2d67353e0059e9a: No bootstrap required, opened a new log
I20260808 22:34:54.322525 25245 tablet_bootstrap.cc:492] T d2f7ff0814894106a1c3da6a1723ef2c P 43172513b66542d79f1d327998eb886c: No bootstrap required, opened a new log
I20260808 22:34:54.322685 25247 ts_tablet_manager.cc:1397] T d2f7ff0814894106a1c3da6a1723ef2c P d868a18f455648d9b2d67353e0059e9a: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 22:34:54.322688 25245 ts_tablet_manager.cc:1397] T d2f7ff0814894106a1c3da6a1723ef2c P 43172513b66542d79f1d327998eb886c: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 22:34:54.325340 25247 raft_consensus.cc:348] T d2f7ff0814894106a1c3da6a1723ef2c P d868a18f455648d9b2d67353e0059e9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d868a18f455648d9b2d67353e0059e9a" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 40277 } } peers { permanent_uuid: "43172513b66542d79f1d327998eb886c" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36833 } } peers { permanent_uuid: "b04503a3ef6e4d6e833b22c13c65122c" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 43459 } }
I20260808 22:34:54.325341 25245 raft_consensus.cc:348] T d2f7ff0814894106a1c3da6a1723ef2c P 43172513b66542d79f1d327998eb886c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d868a18f455648d9b2d67353e0059e9a" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 40277 } } peers { permanent_uuid: "43172513b66542d79f1d327998eb886c" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36833 } } peers { permanent_uuid: "b04503a3ef6e4d6e833b22c13c65122c" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 43459 } }
I20260808 22:34:54.325548 25245 raft_consensus.cc:374] T d2f7ff0814894106a1c3da6a1723ef2c P 43172513b66542d79f1d327998eb886c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:34:54.325551 25247 raft_consensus.cc:374] T d2f7ff0814894106a1c3da6a1723ef2c P d868a18f455648d9b2d67353e0059e9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:34:54.325583 25245 raft_consensus.cc:729] T d2f7ff0814894106a1c3da6a1723ef2c P 43172513b66542d79f1d327998eb886c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43172513b66542d79f1d327998eb886c, State: Initialized, Role: FOLLOWER
I20260808 22:34:54.325606 25247 raft_consensus.cc:729] T d2f7ff0814894106a1c3da6a1723ef2c P d868a18f455648d9b2d67353e0059e9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d868a18f455648d9b2d67353e0059e9a, State: Initialized, Role: FOLLOWER
I20260808 22:34:54.326011 25245 consensus_queue.cc:260] T d2f7ff0814894106a1c3da6a1723ef2c P 43172513b66542d79f1d327998eb886c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d868a18f455648d9b2d67353e0059e9a" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 40277 } } peers { permanent_uuid: "43172513b66542d79f1d327998eb886c" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36833 } } peers { permanent_uuid: "b04503a3ef6e4d6e833b22c13c65122c" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 43459 } }
I20260808 22:34:54.326037 25247 consensus_queue.cc:260] T d2f7ff0814894106a1c3da6a1723ef2c P d868a18f455648d9b2d67353e0059e9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d868a18f455648d9b2d67353e0059e9a" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 40277 } } peers { permanent_uuid: "43172513b66542d79f1d327998eb886c" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36833 } } peers { permanent_uuid: "b04503a3ef6e4d6e833b22c13c65122c" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 43459 } }
I20260808 22:34:54.326761 25014 heartbeater.cc:503] Master 127.31.87.190:37545 was elected leader, sending a full tablet report...
I20260808 22:34:54.326936 25247 ts_tablet_manager.cc:1428] T d2f7ff0814894106a1c3da6a1723ef2c P d868a18f455648d9b2d67353e0059e9a: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260808 22:34:54.326943 25245 ts_tablet_manager.cc:1428] T d2f7ff0814894106a1c3da6a1723ef2c P 43172513b66542d79f1d327998eb886c: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260808 22:34:54.327251 25227 heartbeater.cc:503] Master 127.31.87.190:37545 was elected leader, sending a full tablet report...
I20260808 22:34:54.344497 24804 raft_consensus.cc:482] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:34:54.344671 24804 raft_consensus.cc:504] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d868a18f455648d9b2d67353e0059e9a" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 40277 } } peers { permanent_uuid: "43172513b66542d79f1d327998eb886c" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36833 } } peers { permanent_uuid: "b04503a3ef6e4d6e833b22c13c65122c" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 43459 } }
I20260808 22:34:54.345727 24804 leader_election.cc:288] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d868a18f455648d9b2d67353e0059e9a (127.31.87.130:40277), 43172513b66542d79f1d327998eb886c (127.31.87.131:36833)
I20260808 22:34:54.349452 24798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:54.345937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.130:40277 (local address 127.31.87.129:59305)
0808 22:34:54.346065 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:54.346080 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:34:54.346172 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:34:54.346489 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:34:54.346492 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:34:54.346513 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:34:54.346624 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:54.346631 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:54.347426 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:54.347430 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:54.348274 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:54.348284 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:54.348986 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:54.348990 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:54.349120 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:54.349171 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 22:34:54.349245 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 22:34:54.349329 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260808 22:34:54.349450 25242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:54.345995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:59305 (local address 127.31.87.130:40277)
0808 22:34:54.346125 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:54.346128 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:54.346140 (+ 12us) server_negotiation.cc:407] Connection header received
0808 22:34:54.346298 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:54.346301 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:54.346349 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:54.346426 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:34:54.346735 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:54.347330 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:54.348390 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:54.348890 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:54.348974 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:54.349013 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:54.349266 (+ 253us) server_negotiation.cc:299] Negotiation successful
0808 22:34:54.349331 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 22:34:54.349792 25243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:54.346295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:52339 (local address 127.31.87.131:36833)
0808 22:34:54.346421 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:54.346425 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:54.346628 (+ 203us) server_negotiation.cc:407] Connection header received
0808 22:34:54.346721 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:54.346724 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:54.346773 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:54.346822 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:34:54.347072 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:54.347601 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:54.348740 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:54.349241 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:54.349334 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:54.349374 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:54.349613 (+ 239us) server_negotiation.cc:299] Negotiation successful
0808 22:34:54.349688 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 22:34:54.350028 24942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f7ff0814894106a1c3da6a1723ef2c" candidate_uuid: "b04503a3ef6e4d6e833b22c13c65122c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d868a18f455648d9b2d67353e0059e9a" is_pre_election: true
I20260808 22:34:54.350095 25259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:54.346216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.131:36833 (local address 127.31.87.129:52339)
0808 22:34:54.346543 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:54.346555 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:34:54.346638 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:34:54.346846 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:34:54.346849 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:34:54.346865 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:34:54.346981 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:54.346987 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:54.347700 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:54.347702 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:54.348620 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:34:54.348631 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:34:54.349371 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:34:54.349375 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:34:54.349463 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:54.349488 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 22:34:54.349918 (+ 430us) client_negotiation.cc:240] Negotiation successful
0808 22:34:54.349992 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":2,"thread_start_us":118,"threads_started":1}
I20260808 22:34:54.350288 24942 raft_consensus.cc:2454] T d2f7ff0814894106a1c3da6a1723ef2c P d868a18f455648d9b2d67353e0059e9a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b04503a3ef6e4d6e833b22c13c65122c in term 0.
I20260808 22:34:54.350687 25153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f7ff0814894106a1c3da6a1723ef2c" candidate_uuid: "b04503a3ef6e4d6e833b22c13c65122c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43172513b66542d79f1d327998eb886c" is_pre_election: true
I20260808 22:34:54.350718 24611 leader_election.cc:302] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b04503a3ef6e4d6e833b22c13c65122c, d868a18f455648d9b2d67353e0059e9a; no voters:
I20260808 22:34:54.350921 25153 raft_consensus.cc:2454] T d2f7ff0814894106a1c3da6a1723ef2c P 43172513b66542d79f1d327998eb886c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b04503a3ef6e4d6e833b22c13c65122c in term 0.
I20260808 22:34:54.350931 24804 raft_consensus.cc:2781] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:34:54.351006 24804 raft_consensus.cc:482] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:34:54.351044 24804 raft_consensus.cc:3037] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:34:54.352160 24804 raft_consensus.cc:504] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d868a18f455648d9b2d67353e0059e9a" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 40277 } } peers { permanent_uuid: "43172513b66542d79f1d327998eb886c" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36833 } } peers { permanent_uuid: "b04503a3ef6e4d6e833b22c13c65122c" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 43459 } }
I20260808 22:34:54.352448 24804 leader_election.cc:288] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [CANDIDATE]: Term 1 election: Requested vote from peers d868a18f455648d9b2d67353e0059e9a (127.31.87.130:40277), 43172513b66542d79f1d327998eb886c (127.31.87.131:36833)
I20260808 22:34:54.352749 24942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f7ff0814894106a1c3da6a1723ef2c" candidate_uuid: "b04503a3ef6e4d6e833b22c13c65122c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d868a18f455648d9b2d67353e0059e9a"
I20260808 22:34:54.352805 25153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f7ff0814894106a1c3da6a1723ef2c" candidate_uuid: "b04503a3ef6e4d6e833b22c13c65122c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43172513b66542d79f1d327998eb886c"
I20260808 22:34:54.352854 24942 raft_consensus.cc:3037] T d2f7ff0814894106a1c3da6a1723ef2c P d868a18f455648d9b2d67353e0059e9a [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:34:54.352895 25153 raft_consensus.cc:3037] T d2f7ff0814894106a1c3da6a1723ef2c P 43172513b66542d79f1d327998eb886c [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:34:54.353915 25153 raft_consensus.cc:2454] T d2f7ff0814894106a1c3da6a1723ef2c P 43172513b66542d79f1d327998eb886c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b04503a3ef6e4d6e833b22c13c65122c in term 1.
I20260808 22:34:54.353948 24942 raft_consensus.cc:2454] T d2f7ff0814894106a1c3da6a1723ef2c P d868a18f455648d9b2d67353e0059e9a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b04503a3ef6e4d6e833b22c13c65122c in term 1.
I20260808 22:34:54.354199 24611 leader_election.cc:302] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43172513b66542d79f1d327998eb886c, b04503a3ef6e4d6e833b22c13c65122c; no voters:
I20260808 22:34:54.354375 24804 raft_consensus.cc:2781] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:34:54.354570 24804 raft_consensus.cc:686] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [term 1 LEADER]: Becoming Leader. State: Replica: b04503a3ef6e4d6e833b22c13c65122c, State: Running, Role: LEADER
I20260808 22:34:54.354985 24804 consensus_queue.cc:237] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d868a18f455648d9b2d67353e0059e9a" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 40277 } } peers { permanent_uuid: "43172513b66542d79f1d327998eb886c" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36833 } } peers { permanent_uuid: "b04503a3ef6e4d6e833b22c13c65122c" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 43459 } }
I20260808 22:34:54.357553 24488 catalog_manager.cc:5495] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c reported cstate change: term changed from 0 to 1, leader changed from <none> to b04503a3ef6e4d6e833b22c13c65122c (127.31.87.129). New cstate: current_term: 1 leader_uuid: "b04503a3ef6e4d6e833b22c13c65122c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d868a18f455648d9b2d67353e0059e9a" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 40277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43172513b66542d79f1d327998eb886c" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36833 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b04503a3ef6e4d6e833b22c13c65122c" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 43459 } health_report { overall_health: HEALTHY } } }
I20260808 22:34:54.371177 24806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:54.368119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45884 (local address 127.31.87.129:43459)
0808 22:34:54.368255 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 22:34:54.368259 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:34:54.368331 (+ 72us) server_negotiation.cc:407] Connection header received
0808 22:34:54.368469 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:34:54.368471 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:34:54.368519 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:34:54.368593 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:34:54.368752 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:54.369304 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:54.370030 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:34:54.370257 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:34:54.370325 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:34:54.370505 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:34:54.370870 (+ 365us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:34:54.370948 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 22:34:54.370991 (+ 43us) server_negotiation.cc:299] Negotiation successful
0808 22:34:54.371036 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 22:34:54.374526 24942 raft_consensus.cc:1261] T d2f7ff0814894106a1c3da6a1723ef2c P d868a18f455648d9b2d67353e0059e9a [term 1 FOLLOWER]: Refusing update from remote peer b04503a3ef6e4d6e833b22c13c65122c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:34:54.374526 25153 raft_consensus.cc:1261] T d2f7ff0814894106a1c3da6a1723ef2c P 43172513b66542d79f1d327998eb886c [term 1 FOLLOWER]: Refusing update from remote peer b04503a3ef6e4d6e833b22c13c65122c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:34:54.375121 24804 consensus_queue.cc:1035] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [LEADER]: Connected to new peer: Peer: permanent_uuid: "43172513b66542d79f1d327998eb886c" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:34:54.375308 25260 consensus_queue.cc:1035] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d868a18f455648d9b2d67353e0059e9a" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 40277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:34:54.389793 24485 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33698:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:34:54.389926 24485 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33698:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:34:54.392055 24485 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8: Sending DeleteTablet for 3 replicas of tablet d2f7ff0814894106a1c3da6a1723ef2c
I20260808 22:34:54.392635 24909 tablet_service.cc:1514] Processing DeleteTablet for tablet d2f7ff0814894106a1c3da6a1723ef2c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:34:54 UTC) from {username='kudu', principal='kudu/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:58664
I20260808 22:34:54.392726 24681 tablet_service.cc:1514] Processing DeleteTablet for tablet d2f7ff0814894106a1c3da6a1723ef2c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:34:54 UTC) from {username='kudu', principal='kudu/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:45868
I20260808 22:34:54.392962 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 24548
I20260808 22:34:54.393265 25127 tablet_service.cc:1514] Processing DeleteTablet for tablet d2f7ff0814894106a1c3da6a1723ef2c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:34:54 UTC) from {username='kudu', principal='kudu/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:42792
I20260808 22:34:54.400102 25279 tablet_replica.cc:330] T d2f7ff0814894106a1c3da6a1723ef2c P d868a18f455648d9b2d67353e0059e9a: stopping tablet replica
I20260808 22:34:54.400489 25279 raft_consensus.cc:2229] T d2f7ff0814894106a1c3da6a1723ef2c P d868a18f455648d9b2d67353e0059e9a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:34:54.400879 25279 raft_consensus.cc:2258] T d2f7ff0814894106a1c3da6a1723ef2c P d868a18f455648d9b2d67353e0059e9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:34:54.403810 25279 ts_tablet_manager.cc:1905] T d2f7ff0814894106a1c3da6a1723ef2c P d868a18f455648d9b2d67353e0059e9a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:34:54.406332 25279 ts_tablet_manager.cc:1918] T d2f7ff0814894106a1c3da6a1723ef2c P d868a18f455648d9b2d67353e0059e9a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:34:54.406421 25279 log.cc:1196] T d2f7ff0814894106a1c3da6a1723ef2c P d868a18f455648d9b2d67353e0059e9a: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-1/wal/wals/d2f7ff0814894106a1c3da6a1723ef2c
I20260808 22:34:54.406723 25279 ts_tablet_manager.cc:1939] T d2f7ff0814894106a1c3da6a1723ef2c P d868a18f455648d9b2d67353e0059e9a: Deleting consensus metadata
I20260808 22:34:54.407774 24463 catalog_manager.cc:4841] TS d868a18f455648d9b2d67353e0059e9a (127.31.87.130:40277): tablet d2f7ff0814894106a1c3da6a1723ef2c (table test-table [id=113bbc9d1c944860823b4eaeddfbb852]) successfully deleted
I20260808 22:34:54.410193 25282 tablet_replica.cc:330] T d2f7ff0814894106a1c3da6a1723ef2c P 43172513b66542d79f1d327998eb886c: stopping tablet replica
I20260808 22:34:54.410467 25282 raft_consensus.cc:2229] T d2f7ff0814894106a1c3da6a1723ef2c P 43172513b66542d79f1d327998eb886c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:34:54.410739 25282 raft_consensus.cc:2258] T d2f7ff0814894106a1c3da6a1723ef2c P 43172513b66542d79f1d327998eb886c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:34:54.411901 25282 ts_tablet_manager.cc:1905] T d2f7ff0814894106a1c3da6a1723ef2c P 43172513b66542d79f1d327998eb886c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:34:54.413973 25282 ts_tablet_manager.cc:1918] T d2f7ff0814894106a1c3da6a1723ef2c P 43172513b66542d79f1d327998eb886c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:34:54.414054 25282 log.cc:1196] T d2f7ff0814894106a1c3da6a1723ef2c P 43172513b66542d79f1d327998eb886c: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-2/wal/wals/d2f7ff0814894106a1c3da6a1723ef2c
I20260808 22:34:54.414327 25282 ts_tablet_manager.cc:1939] T d2f7ff0814894106a1c3da6a1723ef2c P 43172513b66542d79f1d327998eb886c: Deleting consensus metadata
I20260808 22:34:54.415436 24463 catalog_manager.cc:4841] TS 43172513b66542d79f1d327998eb886c (127.31.87.131:36833): tablet d2f7ff0814894106a1c3da6a1723ef2c (table test-table [id=113bbc9d1c944860823b4eaeddfbb852]) successfully deleted
W20260808 22:34:54.415771 24458 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.87.129:43459 (error 108)
I20260808 22:34:54.416041 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 24799
I20260808 22:34:54.421959 24805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:34:54.421649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:43459 (local address 127.0.0.1:45896)
0808 22:34:54.421793 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 22:34:54.421874 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.129:43459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260808 22:34:54.422170 24458 catalog_manager.cc:4584] TS b04503a3ef6e4d6e833b22c13c65122c (127.31.87.129:43459): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d2f7ff0814894106a1c3da6a1723ef2c: Network error: Client connection negotiation failed: client connection to 127.31.87.129:43459: connect: Connection refused (error 111)
I20260808 22:34:54.426411 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 25017
I20260808 22:34:54.448719 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 24435
I20260808 22:35:14.460300 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:37545
--webserver_interface=127.31.87.190
--webserver_port=43477
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:34763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:37545
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:35:14.570155 25285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:35:14.570396 25285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:35:14.570451 25285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:35:14.574343 25285 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 22:35:14.574386 25285 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 22:35:14.574406 25285 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 22:35:14.574424 25285 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:35:14.574456 25285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:35:14.574488 25285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:35:14.574517 25285 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:35:14.574545 25285 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:35:14.579672 25285 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:34763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:37545
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:37545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.87.190
--webserver_port=43477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25285
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:35:14.580586 25285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:35:14.581549 25285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:35:14.589829 25293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:14.589860 25292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:14.589978 25285 server_base.cc:1034] running on GCE node
W20260808 22:35:14.589864 25299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:14.590631 25285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:35:14.591637 25285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:35:14.592796 25285 hybrid_clock.cc:648] HybridClock initialized: now 1786228514592779 us; error 34 us; skew 500 ppm
Aug 08 22:35:14 dist-test-slave-266d krb5kdc[23581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228514, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:35:14.598948 25285 init.cc:377] Logged in from keytab as kudu/127.31.87.190@KRBTEST.COM (short username kudu)
I20260808 22:35:14.599773 25285 webserver.cc:460] Webserver started at http://127.31.87.190:43477/ using document root <none> and password file <none>
I20260808 22:35:14.600278 25285 fs_manager.cc:359] Metadata directory not provided
I20260808 22:35:14.600342 25285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:35:14.603955 25285 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:35:14.605966 25309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:14.607116 25285 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:35:14.607244 25285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "5c4f0c26263f4a79994b0c236d72fad8"
format_stamp: "Formatted at 2026-08-08 22:34:32 on dist-test-slave-266d"
server_key: "f44ab1ebc0fd14d0def43d3ad5410ed0"
server_key_iv: "f97f82f6979387d4b1fdfa875ec3a870"
server_key_version: "encryptionkey@0"
I20260808 22:35:14.607785 25285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:35:14.621896 25285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:35:14.624402 25285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:35:14.624567 25285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:35:14.631078 25285 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.190:37545
I20260808 22:35:14.631352 25384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.190:37545 every 8 connection(s)
I20260808 22:35:14.632051 25285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:35:14.636070 25385 sys_catalog.cc:263] Verifying existing consensus state
I20260808 22:35:14.636827 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 25285
I20260808 22:35:14.637245 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.129:43459
--local_ip_for_outbound_sockets=127.31.87.129
--tserver_master_addrs=127.31.87.190:37545
--webserver_port=45475
--webserver_interface=127.31.87.129
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab
--principal=kudu/127.31.87.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:34763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 22:35:14.637809 25385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8: Bootstrap starting.
I20260808 22:35:14.654533 25385 log.cc:824] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8: Log is configured to *not* fsync() on all Append() calls
I20260808 22:35:14.665508 25385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 22:35:14.665834 25385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8: Bootstrap complete.
I20260808 22:35:14.669590 25385 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c4f0c26263f4a79994b0c236d72fad8" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 37545 } }
I20260808 22:35:14.669965 25385 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c4f0c26263f4a79994b0c236d72fad8, State: Initialized, Role: FOLLOWER
I20260808 22:35:14.670410 25385 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c4f0c26263f4a79994b0c236d72fad8" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 37545 } }
I20260808 22:35:14.670526 25385 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:35:14.670588 25385 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:35:14.670684 25385 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8 [term 2 FOLLOWER]: Advancing to term 3
I20260808 22:35:14.672482 25385 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c4f0c26263f4a79994b0c236d72fad8" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 37545 } }
I20260808 22:35:14.672793 25385 leader_election.cc:302] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c4f0c26263f4a79994b0c236d72fad8; no voters:
I20260808 22:35:14.673066 25385 leader_election.cc:288] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 22:35:14.673209 25394 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 22:35:14.673470 25394 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8 [term 3 LEADER]: Becoming Leader. State: Replica: 5c4f0c26263f4a79994b0c236d72fad8, State: Running, Role: LEADER
I20260808 22:35:14.673832 25394 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c4f0c26263f4a79994b0c236d72fad8" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 37545 } }
I20260808 22:35:14.674234 25385 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:35:14.674762 25394 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c4f0c26263f4a79994b0c236d72fad8. Latest consensus state: current_term: 3 leader_uuid: "5c4f0c26263f4a79994b0c236d72fad8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c4f0c26263f4a79994b0c236d72fad8" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 37545 } } }
I20260808 22:35:14.674877 25394 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8 [sys.catalog]: This master's current role is: LEADER
I20260808 22:35:14.675221 25401 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:35:14.675284 25396 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "5c4f0c26263f4a79994b0c236d72fad8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c4f0c26263f4a79994b0c236d72fad8" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 37545 } } }
I20260808 22:35:14.675423 25396 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8 [sys.catalog]: This master's current role is: LEADER
I20260808 22:35:14.680342 25401 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:35:14.681033 25401 catalog_manager.cc:1259] Loaded cluster ID: d66d2e3ed939478ba6705e1ab06307ff
I20260808 22:35:14.681114 25401 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:35:14.683522 25401 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:35:14.684234 25401 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8: Loaded TSK: 0
I20260808 22:35:14.684324 25401 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5c4f0c26263f4a79994b0c236d72fad8: Loaded TSK: 1
F20260808 22:35:14.685001 25401 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f24c89ad9ad google::LogMessage::Fail() at ??:0
@ 0x7f24c89b088a google::LogMessage::SendToLog() at ??:0
@ 0x7f24c89ad4de google::LogMessage::Flush() at ??:0
@ 0x7f24c89aea19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f24cb67118d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f24cb669f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f24d33127bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f24d32ec99c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f24d32ed178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f24d332edf8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f24d334d82b std::function<>::operator()() at ??:0
@ 0x7f24d32ecb4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f24d32edf3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f24d32ea579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f24d332e640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f24cabff3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f24cabffc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f24cac013f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f24cabf0512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f24cc2346db start_thread at ??:0
@ 0x7f24c74ab61f clone at ??:0
W20260808 22:35:14.767557 25388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:35:14.767802 25388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:35:14.767844 25388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:35:14.772169 25388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:35:14.772213 25388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:35:14.772269 25388 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.129
I20260808 22:35:14.777494 25388 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:34763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab
--principal=kudu/127.31.87.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.129:43459
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.87.129
--webserver_port=45475
--tserver_master_addrs=127.31.87.190:37545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.129
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:35:14.778543 25388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:35:14.779558 25388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:35:14.788800 25469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:14.788812 25474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:14.788820 25465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:14.789428 25388 server_base.cc:1034] running on GCE node
I20260808 22:35:14.789906 25388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:35:14.790660 25388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:35:14.791860 25388 hybrid_clock.cc:648] HybridClock initialized: now 1786228514791828 us; error 40 us; skew 500 ppm
Aug 08 22:35:14 dist-test-slave-266d krb5kdc[23581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228514, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:35:14.794983 25388 init.cc:377] Logged in from keytab as kudu/127.31.87.129@KRBTEST.COM (short username kudu)
I20260808 22:35:14.795804 25388 webserver.cc:460] Webserver started at http://127.31.87.129:45475/ using document root <none> and password file <none>
I20260808 22:35:14.796335 25388 fs_manager.cc:359] Metadata directory not provided
I20260808 22:35:14.796397 25388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:35:14.800103 25388 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:35:14.802209 25483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:14.803476 25388 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:35:14.803589 25388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-0/wal
uuid: "b04503a3ef6e4d6e833b22c13c65122c"
format_stamp: "Formatted at 2026-08-08 22:34:32 on dist-test-slave-266d"
server_key: "d9262e12f89f8cf4cb3863fb25fd7135"
server_key_iv: "7fb3e8b41f829b8c20ae8a72976eefe1"
server_key_version: "encryptionkey@0"
I20260808 22:35:14.804085 25388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:35:14.814411 25388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:35:14.817203 25388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:35:14.817356 25388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:35:14.817862 25388 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:35:14.818858 25495 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 22:35:14.824091 25388 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 22:35:14.824148 25388 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260808 22:35:14.824216 25388 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 22:35:14.825826 25388 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 22:35:14.825870 25388 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260808 22:35:14.826190 25495 tablet_bootstrap.cc:492] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c: Bootstrap starting.
I20260808 22:35:14.835381 25388 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.129:43459
I20260808 22:35:14.835570 25651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.129:43459 every 8 connection(s)
I20260808 22:35:14.836364 25388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
I20260808 22:35:14.844224 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 25388
I20260808 22:35:14.844672 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.130:40277
--local_ip_for_outbound_sockets=127.31.87.130
--tserver_master_addrs=127.31.87.190:37545
--webserver_port=45801
--webserver_interface=127.31.87.130
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab
--principal=kudu/127.31.87.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:34763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 22:35:14.848038 25495 log.cc:824] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c: Log is configured to *not* fsync() on all Append() calls
I20260808 22:35:14.856865 25495 tablet_bootstrap.cc:492] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 22:35:14.857288 25495 tablet_bootstrap.cc:492] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c: Bootstrap complete.
I20260808 22:35:14.858636 25495 ts_tablet_manager.cc:1397] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c: Time spent bootstrapping tablet: real 0.033s user 0.015s sys 0.016s
I20260808 22:35:14.861821 25495 raft_consensus.cc:348] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d868a18f455648d9b2d67353e0059e9a" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 40277 } } peers { permanent_uuid: "43172513b66542d79f1d327998eb886c" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36833 } } peers { permanent_uuid: "b04503a3ef6e4d6e833b22c13c65122c" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 43459 } }
I20260808 22:35:14.862339 25495 raft_consensus.cc:729] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b04503a3ef6e4d6e833b22c13c65122c, State: Initialized, Role: FOLLOWER
I20260808 22:35:14.862901 25495 consensus_queue.cc:260] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d868a18f455648d9b2d67353e0059e9a" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 40277 } } peers { permanent_uuid: "43172513b66542d79f1d327998eb886c" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36833 } } peers { permanent_uuid: "b04503a3ef6e4d6e833b22c13c65122c" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 43459 } }
I20260808 22:35:14.863932 25495 ts_tablet_manager.cc:1428] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c: Time spent starting tablet: real 0.005s user 0.001s sys 0.005s
W20260808 22:35:14.919629 25656 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:35:14.838327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.129:53285)
0808 22:35:14.838818 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:14.838869 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 22:35:14.839643 (+ 774us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:35:14.918876 (+ 79233us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:37545: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":124,"threads_started":1}
W20260808 22:35:14.920303 25652 heartbeater.cc:644] Failed to heartbeat to 127.31.87.190:37545 (0 consecutive failures): Network error: Failed to ping master at 127.31.87.190:37545: Client connection negotiation failed: client connection to 127.31.87.190:37545: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 22:35:14.921061 25656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:14.920809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.129:56729)
0808 22:35:14.920940 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:14.921005 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 22:35:14.921814 25656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:14.921596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.129:35797)
0808 22:35:14.921708 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:14.921756 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 22:35:14.921981 25652 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 22:35:14.957814 25657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:35:14.958024 25657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:35:14.958070 25657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:35:14.961951 25657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:35:14.961999 25657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:35:14.962064 25657 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.130
I20260808 22:35:14.967111 25657 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:34763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab
--principal=kudu/127.31.87.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.130:40277
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.87.130
--webserver_port=45801
--tserver_master_addrs=127.31.87.190:37545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25657
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.130
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:35:14.968000 25657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:35:14.968906 25657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:35:14.977849 25672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:14.977842 25674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:14.977852 25677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:14.978000 25657 server_base.cc:1034] running on GCE node
I20260808 22:35:14.978430 25657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:35:14.979040 25657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:35:14.980185 25657 hybrid_clock.cc:648] HybridClock initialized: now 1786228514980168 us; error 28 us; skew 500 ppm
Aug 08 22:35:14 dist-test-slave-266d krb5kdc[23581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228514, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:35:14.982643 25657 init.cc:377] Logged in from keytab as kudu/127.31.87.130@KRBTEST.COM (short username kudu)
I20260808 22:35:14.983430 25657 webserver.cc:460] Webserver started at http://127.31.87.130:45801/ using document root <none> and password file <none>
I20260808 22:35:14.983928 25657 fs_manager.cc:359] Metadata directory not provided
I20260808 22:35:14.983994 25657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:35:14.987488 25657 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:35:14.989336 25685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:14.990442 25657 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 22:35:14.990541 25657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-1/wal
uuid: "d868a18f455648d9b2d67353e0059e9a"
format_stamp: "Formatted at 2026-08-08 22:34:32 on dist-test-slave-266d"
server_key: "0a91ad264b934d578ca5f97b838f95fc"
server_key_iv: "a30caef0655e0ace7cde4c2a6f13e92a"
server_key_version: "encryptionkey@0"
I20260808 22:35:14.991020 25657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:35:15.002719 25657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:35:15.005168 25657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:35:15.005311 25657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:35:15.005807 25657 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:35:15.006526 25657 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:35:15.006570 25657 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:15.006613 25657 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:35:15.006640 25657 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:15.015106 25657 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.130:40277
I20260808 22:35:15.015142 25866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.130:40277 every 8 connection(s)
I20260808 22:35:15.015998 25657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
I20260808 22:35:15.019271 25871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:15.017916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.130:48051)
0808 22:35:15.018512 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:15.018614 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":106,"threads_started":1}
W20260808 22:35:15.019728 25867 heartbeater.cc:644] Failed to heartbeat to 127.31.87.190:37545 (0 consecutive failures): Network error: Failed to ping master at 127.31.87.190:37545: Client connection negotiation failed: client connection to 127.31.87.190:37545: connect: Connection refused (error 111)
I20260808 22:35:15.020237 25871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:15.020039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.130:49969)
0808 22:35:15.020139 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:15.020184 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 22:35:15.020763 25871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:15.020603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.130:55617)
0808 22:35:15.020679 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:15.020718 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260808 22:35:15.020905 25867 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 22:35:15.023228 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 25657
I20260808 22:35:15.023595 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.131:36833
--local_ip_for_outbound_sockets=127.31.87.131
--tserver_master_addrs=127.31.87.190:37545
--webserver_port=35617
--webserver_interface=127.31.87.131
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab
--principal=kudu/127.31.87.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:34763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:35:15.134128 25872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:35:15.134372 25872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:35:15.134414 25872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:35:15.138327 25872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:35:15.138382 25872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:35:15.138447 25872 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.131
I20260808 22:35:15.143414 25872 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:34763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab
--principal=kudu/127.31.87.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.131:36833
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.87.131
--webserver_port=35617
--tserver_master_addrs=127.31.87.190:37545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25872
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.131
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:35:15.144332 25872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:35:15.145285 25872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:35:15.154533 25879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:15.154531 25880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:15.154616 25872 server_base.cc:1034] running on GCE node
W20260808 22:35:15.154528 25882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:15.155201 25872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:35:15.155807 25872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:35:15.156987 25872 hybrid_clock.cc:648] HybridClock initialized: now 1786228515156966 us; error 34 us; skew 500 ppm
Aug 08 22:35:15 dist-test-slave-266d krb5kdc[23581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228515, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:35:15.159744 25872 init.cc:377] Logged in from keytab as kudu/127.31.87.131@KRBTEST.COM (short username kudu)
I20260808 22:35:15.160605 25872 webserver.cc:460] Webserver started at http://127.31.87.131:35617/ using document root <none> and password file <none>
I20260808 22:35:15.161165 25872 fs_manager.cc:359] Metadata directory not provided
I20260808 22:35:15.161238 25872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:35:15.164915 25872 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:35:15.166994 25894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:15.168184 25872 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:35:15.168298 25872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-2/wal
uuid: "43172513b66542d79f1d327998eb886c"
format_stamp: "Formatted at 2026-08-08 22:34:33 on dist-test-slave-266d"
server_key: "b28fe866acfd0353c226c51e0002da45"
server_key_iv: "13c50f9463f0a73c1d1ed778c9b16ed4"
server_key_version: "encryptionkey@0"
I20260808 22:35:15.168840 25872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:35:15.181669 25872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:35:15.184556 25872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:35:15.184726 25872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:35:15.185298 25872 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:35:15.186132 25872 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:35:15.186179 25872 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:15.186223 25872 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:35:15.186252 25872 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:15.194506 25872 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.131:36833
I20260808 22:35:15.194550 26063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.131:36833 every 8 connection(s)
I20260808 22:35:15.195467 25872 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
I20260808 22:35:15.198906 26074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:15.197498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.131:41909)
0808 22:35:15.198153 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:15.198252 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":120,"threads_started":1}
W20260808 22:35:15.199313 26066 heartbeater.cc:644] Failed to heartbeat to 127.31.87.190:37545 (0 consecutive failures): Network error: Failed to ping master at 127.31.87.190:37545: Client connection negotiation failed: client connection to 127.31.87.190:37545: connect: Connection refused (error 111)
I20260808 22:35:15.199858 26074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:15.199651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.131:54097)
0808 22:35:15.199760 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:15.199806 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 22:35:15.199873 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 25872
I20260808 22:35:15.200407 26074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:15.200213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.131:40795)
0808 22:35:15.200307 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:15.200353 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260808 22:35:15.200587 26066 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 22:35:15.922902 26076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:15.922488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.129:37871)
0808 22:35:15.922730 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:15.922783 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":76,"threads_started":1}
I20260808 22:35:16.021826 26077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:16.021414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.130:46835)
0808 22:35:16.021674 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:16.021728 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":89,"threads_started":1}
I20260808 22:35:16.201411 26078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:16.201038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.131:57367)
0808 22:35:16.201262 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:16.201314 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":83,"threads_started":1}
I20260808 22:35:16.257933 26080 raft_consensus.cc:482] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:35:16.258133 26080 raft_consensus.cc:504] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d868a18f455648d9b2d67353e0059e9a" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 40277 } } peers { permanent_uuid: "43172513b66542d79f1d327998eb886c" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36833 } } peers { permanent_uuid: "b04503a3ef6e4d6e833b22c13c65122c" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 43459 } }
I20260808 22:35:16.259234 26080 leader_election.cc:288] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d868a18f455648d9b2d67353e0059e9a (127.31.87.130:40277), 43172513b66542d79f1d327998eb886c (127.31.87.131:36833)
Aug 08 22:35:16 dist-test-slave-266d krb5kdc[23581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228514, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.129@KRBTEST.COM for kudu/127.31.87.131@KRBTEST.COM
Aug 08 22:35:16 dist-test-slave-266d krb5kdc[23581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228514, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.129@KRBTEST.COM for kudu/127.31.87.130@KRBTEST.COM
I20260808 22:35:16.274708 26076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:16.259444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.130:40277 (local address 127.31.87.129:52713)
0808 22:35:16.259558 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:16.259572 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:35:16.259676 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:35:16.261806 (+ 2130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:35:16.261812 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:35:16.262413 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:35:16.263216 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:16.263237 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:16.265129 (+ 1892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:16.265132 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:16.266626 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:16.266638 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:16.267565 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:16.267568 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:16.267890 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:16.268557 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:35:16.268579 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:35:16.270619 (+ 2040us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:35:16.272608 (+ 1989us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:35:16.272611 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:35:16.272612 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:35:16.272842 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:35:16.273197 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:35:16.273200 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:35:16.273202 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:35:16.273309 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:35:16.273998 (+ 689us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:35:16.274004 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:35:16.274224 (+ 220us) client_negotiation.cc:769] Sending connection context
0808 22:35:16.274371 (+ 147us) client_negotiation.cc:240] Negotiation successful
0808 22:35:16.274571 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":275,"spinlock_wait_cycles":50688}
I20260808 22:35:16.274814 26081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:16.259574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:52713 (local address 127.31.87.130:40277)
0808 22:35:16.259953 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:16.259975 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:16.260014 (+ 39us) server_negotiation.cc:407] Connection header received
0808 22:35:16.260901 (+ 887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:16.260910 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:16.261378 (+ 468us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:16.261695 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:16.263415 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:16.264948 (+ 1533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:16.266734 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:16.267467 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:16.267605 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:16.270785 (+ 3180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:16.270790 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:16.270800 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:35:16.270827 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:16.272500 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:35:16.272943 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:35:16.272948 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:35:16.272952 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:35:16.273066 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:35:16.273420 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:35:16.273423 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:35:16.273425 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:35:16.273886 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:16.273989 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:16.274458 (+ 469us) server_negotiation.cc:299] Negotiation successful
0808 22:35:16.274665 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":93,"threads_started":1}
I20260808 22:35:16.275250 26082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:16.259704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.131:36833 (local address 127.31.87.129:58433)
0808 22:35:16.260003 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:16.260018 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:35:16.260123 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:35:16.261806 (+ 1683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:35:16.261812 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:35:16.262413 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:35:16.263216 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:16.263237 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:16.265132 (+ 1895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:16.265135 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:16.266626 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:16.266638 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:16.267553 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:16.267557 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:16.267890 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:16.268544 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:35:16.268566 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:35:16.270274 (+ 1708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:35:16.272381 (+ 2107us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:35:16.272388 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:35:16.272402 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:35:16.272732 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:35:16.273111 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:35:16.273114 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:35:16.273116 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:35:16.273283 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:35:16.274845 (+ 1562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:35:16.274849 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:35:16.274936 (+ 87us) client_negotiation.cc:769] Sending connection context
0808 22:35:16.275023 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 22:35:16.275140 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"mutex_wait_us":280,"spinlock_wait_cycles":9600,"thread_start_us":79,"threads_started":1}
I20260808 22:35:16.275823 26085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:16.259889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:58433 (local address 127.31.87.131:36833)
0808 22:35:16.260211 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:16.260232 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:16.260277 (+ 45us) server_negotiation.cc:407] Connection header received
0808 22:35:16.261006 (+ 729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:16.261013 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:16.261377 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:16.261695 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:16.263415 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:16.264948 (+ 1533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:16.266734 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:16.267440 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:16.267607 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:16.270496 (+ 2889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:16.270502 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:16.270518 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:35:16.270551 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:16.272275 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:35:16.272864 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:35:16.272871 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:35:16.272876 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:35:16.273004 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:35:16.273413 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:35:16.273416 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:35:16.273418 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:35:16.273888 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:16.274008 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:16.275496 (+ 1488us) server_negotiation.cc:299] Negotiation successful
0808 22:35:16.275696 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":119,"threads_started":1}
W20260808 22:35:16.277237 25489 leader_election.cc:341] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d868a18f455648d9b2d67353e0059e9a (127.31.87.130:40277): Not found: Tablet not found: d2f7ff0814894106a1c3da6a1723ef2c
W20260808 22:35:16.277872 25489 leader_election.cc:341] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 43172513b66542d79f1d327998eb886c (127.31.87.131:36833): Not found: Tablet not found: d2f7ff0814894106a1c3da6a1723ef2c
I20260808 22:35:16.277947 25489 leader_election.cc:302] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b04503a3ef6e4d6e833b22c13c65122c; no voters: 43172513b66542d79f1d327998eb886c, d868a18f455648d9b2d67353e0059e9a
I20260808 22:35:16.278198 26080 raft_consensus.cc:2726] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 22:35:16.923950 26086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:16.923534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.129:40341)
0808 22:35:16.923793 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:16.923848 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":82,"threads_started":1}
I20260808 22:35:17.022833 26087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:17.022447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.130:53401)
0808 22:35:17.022696 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:17.022748 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":87,"threads_started":1}
I20260808 22:35:17.202324 26088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:17.201971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.131:52385)
0808 22:35:17.202194 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:17.202243 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":68,"threads_started":1}
I20260808 22:35:17.825302 26089 raft_consensus.cc:482] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:35:17.825433 26089 raft_consensus.cc:504] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d868a18f455648d9b2d67353e0059e9a" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 40277 } } peers { permanent_uuid: "43172513b66542d79f1d327998eb886c" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36833 } } peers { permanent_uuid: "b04503a3ef6e4d6e833b22c13c65122c" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 43459 } }
I20260808 22:35:17.825824 26089 leader_election.cc:288] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d868a18f455648d9b2d67353e0059e9a (127.31.87.130:40277), 43172513b66542d79f1d327998eb886c (127.31.87.131:36833)
W20260808 22:35:17.826655 25489 leader_election.cc:341] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d868a18f455648d9b2d67353e0059e9a (127.31.87.130:40277): Not found: Tablet not found: d2f7ff0814894106a1c3da6a1723ef2c
W20260808 22:35:17.826783 25489 leader_election.cc:341] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 43172513b66542d79f1d327998eb886c (127.31.87.131:36833): Not found: Tablet not found: d2f7ff0814894106a1c3da6a1723ef2c
I20260808 22:35:17.826836 25489 leader_election.cc:302] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b04503a3ef6e4d6e833b22c13c65122c; no voters: 43172513b66542d79f1d327998eb886c, d868a18f455648d9b2d67353e0059e9a
I20260808 22:35:17.826988 26089 raft_consensus.cc:2726] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 22:35:17.925163 26090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:17.924663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.129:39485)
0808 22:35:17.924938 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:17.925002 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":77,"threads_started":1}
I20260808 22:35:18.023878 26091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:18.023475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.130:57993)
0808 22:35:18.023716 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:18.023772 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":75,"threads_started":1}
I20260808 22:35:18.203433 26093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:18.203054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.131:56467)
0808 22:35:18.203283 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:18.203334 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":74,"threads_started":1}
I20260808 22:35:18.926249 26095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:18.925859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.129:33363)
0808 22:35:18.926084 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:18.926148 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":73,"threads_started":1}
I20260808 22:35:19.024870 26096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:19.024494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.130:56173)
0808 22:35:19.024732 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:19.024782 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":84,"threads_started":1}
I20260808 22:35:19.204546 26100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:19.204136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.131:35109)
0808 22:35:19.204391 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:19.204446 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":78,"threads_started":1}
I20260808 22:35:19.927515 26101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:19.927046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.129:32847)
0808 22:35:19.927361 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:19.927415 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":79,"threads_started":1}
I20260808 22:35:20.025920 26102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:20.025514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.130:56729)
0808 22:35:20.025766 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:20.025821 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":72,"threads_started":1}
I20260808 22:35:20.206272 26105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:20.205813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.131:59771)
0808 22:35:20.206093 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:20.206164 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":84,"threads_started":1}
I20260808 22:35:20.928615 26106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:20.928210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.129:47781)
0808 22:35:20.928463 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:20.928516 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":86,"threads_started":1}
I20260808 22:35:21.027043 26107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:21.026607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.130:44933)
0808 22:35:21.026871 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:21.026935 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":74,"threads_started":1}
I20260808 22:35:21.112793 26108 raft_consensus.cc:482] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:35:21.112924 26108 raft_consensus.cc:504] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d868a18f455648d9b2d67353e0059e9a" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 40277 } } peers { permanent_uuid: "43172513b66542d79f1d327998eb886c" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36833 } } peers { permanent_uuid: "b04503a3ef6e4d6e833b22c13c65122c" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 43459 } }
I20260808 22:35:21.113359 26108 leader_election.cc:288] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d868a18f455648d9b2d67353e0059e9a (127.31.87.130:40277), 43172513b66542d79f1d327998eb886c (127.31.87.131:36833)
W20260808 22:35:21.114127 25489 leader_election.cc:341] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d868a18f455648d9b2d67353e0059e9a (127.31.87.130:40277): Not found: Tablet not found: d2f7ff0814894106a1c3da6a1723ef2c
W20260808 22:35:21.114264 25489 leader_election.cc:341] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 43172513b66542d79f1d327998eb886c (127.31.87.131:36833): Not found: Tablet not found: d2f7ff0814894106a1c3da6a1723ef2c
I20260808 22:35:21.114310 25489 leader_election.cc:302] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b04503a3ef6e4d6e833b22c13c65122c; no voters: 43172513b66542d79f1d327998eb886c, d868a18f455648d9b2d67353e0059e9a
I20260808 22:35:21.114463 26108 raft_consensus.cc:2726] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 22:35:21.207304 26112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:21.206937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.131:58221)
0808 22:35:21.207163 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:21.207216 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":70,"threads_started":1}
I20260808 22:35:21.929699 26113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:21.929307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.129:47425)
0808 22:35:21.929548 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:21.929600 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":74,"threads_started":1}
I20260808 22:35:22.028102 26114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:22.027679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.130:53275)
0808 22:35:22.027959 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:22.028010 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":71,"threads_started":1}
I20260808 22:35:22.208343 26116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:22.207924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.131:44179)
0808 22:35:22.208199 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:22.208253 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":69,"threads_started":1}
I20260808 22:35:22.930966 26117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:22.930515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.129:47111)
0808 22:35:22.930802 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:22.930866 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":81,"threads_started":1}
I20260808 22:35:23.029307 26118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:23.028855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.130:47961)
0808 22:35:23.029139 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:23.029206 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":76,"threads_started":1}
I20260808 22:35:23.209539 26119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:23.209072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.131:52283)
0808 22:35:23.209383 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:23.209448 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":84,"threads_started":1}
I20260808 22:35:23.932144 26120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:23.931712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.129:42791)
0808 22:35:23.931989 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:23.932044 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":96,"threads_started":1}
I20260808 22:35:24.030510 26121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:24.030018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.130:36119)
0808 22:35:24.030320 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:24.030387 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":94,"threads_started":1}
I20260808 22:35:24.210534 26122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:24.210160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.131:59183)
0808 22:35:24.210395 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:24.210444 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":74,"threads_started":1}
I20260808 22:35:24.365733 26123 raft_consensus.cc:482] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:35:24.365885 26123 raft_consensus.cc:504] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d868a18f455648d9b2d67353e0059e9a" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 40277 } } peers { permanent_uuid: "43172513b66542d79f1d327998eb886c" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36833 } } peers { permanent_uuid: "b04503a3ef6e4d6e833b22c13c65122c" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 43459 } }
I20260808 22:35:24.366302 26123 leader_election.cc:288] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d868a18f455648d9b2d67353e0059e9a (127.31.87.130:40277), 43172513b66542d79f1d327998eb886c (127.31.87.131:36833)
W20260808 22:35:24.367069 25489 leader_election.cc:341] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d868a18f455648d9b2d67353e0059e9a (127.31.87.130:40277): Not found: Tablet not found: d2f7ff0814894106a1c3da6a1723ef2c
W20260808 22:35:24.367192 25489 leader_election.cc:341] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 43172513b66542d79f1d327998eb886c (127.31.87.131:36833): Not found: Tablet not found: d2f7ff0814894106a1c3da6a1723ef2c
I20260808 22:35:24.367237 25489 leader_election.cc:302] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b04503a3ef6e4d6e833b22c13c65122c; no voters: 43172513b66542d79f1d327998eb886c, d868a18f455648d9b2d67353e0059e9a
I20260808 22:35:24.367405 26123 raft_consensus.cc:2726] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 22:35:24.933444 26124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:24.933014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.129:59347)
0808 22:35:24.933279 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:24.933338 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":105,"threads_started":1}
I20260808 22:35:25.031667 26126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:25.031286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.130:57807)
0808 22:35:25.031517 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:25.031572 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":79,"threads_started":1}
I20260808 22:35:25.211520 26129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:25.211170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.131:40059)
0808 22:35:25.211380 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:25.211432 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":80,"threads_started":1}
I20260808 22:35:25.934518 26130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:25.934101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.129:59577)
0808 22:35:25.934361 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:25.934417 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":71,"threads_started":1}
I20260808 22:35:26.032711 26131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:26.032342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.130:50947)
0808 22:35:26.032566 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:26.032618 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":72,"threads_started":1}
I20260808 22:35:26.212548 26133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:26.212167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.131:56197)
0808 22:35:26.212409 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:26.212460 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":82,"threads_started":1}
I20260808 22:35:26.935588 26134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:26.935188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.129:60651)
0808 22:35:26.935432 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:26.935487 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":84,"threads_started":1}
I20260808 22:35:27.033723 26135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:27.033355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.130:43173)
0808 22:35:27.033577 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:27.033627 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":71,"threads_started":1}
I20260808 22:35:27.213718 26136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:27.213340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.131:55835)
0808 22:35:27.213575 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:27.213625 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":72,"threads_started":1}
I20260808 22:35:27.936648 26137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:27.936214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.129:33369)
0808 22:35:27.936493 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:27.936547 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":90,"threads_started":1}
I20260808 22:35:28.034792 26139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:28.034371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.130:42399)
0808 22:35:28.034629 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:28.034684 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":91,"threads_started":1}
I20260808 22:35:28.214715 26141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:28.214341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.131:32901)
0808 22:35:28.214577 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:28.214627 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":84,"threads_started":1}
I20260808 22:35:28.848002 26142 raft_consensus.cc:482] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:35:28.848145 26142 raft_consensus.cc:504] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d868a18f455648d9b2d67353e0059e9a" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 40277 } } peers { permanent_uuid: "43172513b66542d79f1d327998eb886c" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 36833 } } peers { permanent_uuid: "b04503a3ef6e4d6e833b22c13c65122c" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 43459 } }
I20260808 22:35:28.848542 26142 leader_election.cc:288] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d868a18f455648d9b2d67353e0059e9a (127.31.87.130:40277), 43172513b66542d79f1d327998eb886c (127.31.87.131:36833)
W20260808 22:35:28.849318 25489 leader_election.cc:341] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d868a18f455648d9b2d67353e0059e9a (127.31.87.130:40277): Not found: Tablet not found: d2f7ff0814894106a1c3da6a1723ef2c
W20260808 22:35:28.849447 25489 leader_election.cc:341] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 43172513b66542d79f1d327998eb886c (127.31.87.131:36833): Not found: Tablet not found: d2f7ff0814894106a1c3da6a1723ef2c
I20260808 22:35:28.849485 25489 leader_election.cc:302] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b04503a3ef6e4d6e833b22c13c65122c; no voters: 43172513b66542d79f1d327998eb886c, d868a18f455648d9b2d67353e0059e9a
I20260808 22:35:28.849632 26142 raft_consensus.cc:2726] T d2f7ff0814894106a1c3da6a1723ef2c P b04503a3ef6e4d6e833b22c13c65122c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 22:35:28.937783 26147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:28.937361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.129:41963)
0808 22:35:28.937623 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:28.937680 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":75,"threads_started":1}
I20260808 22:35:29.036063 26148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:29.035590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.130:55347)
0808 22:35:29.035897 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:29.035962 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":95,"threads_started":1}
I20260808 22:35:29.215914 26151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:29.215473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.131:44073)
0808 22:35:29.215748 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:29.215820 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":98,"threads_started":1}
I20260808 22:35:29.939217 26153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:29.938796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.129:45341)
0808 22:35:29.939050 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:29.939115 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":76,"threads_started":1}
I20260808 22:35:30.037249 26154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:30.036790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.130:34847)
0808 22:35:30.037060 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:30.037148 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":88,"threads_started":1}
I20260808 22:35:30.205077 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 25388
I20260808 22:35:30.217191 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 25657
I20260808 22:35:30.217253 26155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:30.216740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:37545 (local address 127.31.87.131:54891)
0808 22:35:30.217084 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:30.217158 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.190:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":72,"threads_started":1}
I20260808 22:35:30.227519 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 25872
2026-08-08T22:35:30Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59169 ms)
[----------] 30 tests from SecurityITest (239197 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:35:30 dist-test-slave-266d krb5kdc[26158](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:35:30 dist-test-slave-266d krb5kdc[26158](info): set up 2 sockets
Aug 08 22:35:30 dist-test-slave-266d krb5kdc[26158](info): commencing operation
krb5kdc: starting...
W20260808 22:35:31.365777 32094 mini_kdc.cc:121] Time spent starting KDC: real 1.110s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Aug 08 22:35:31 dist-test-slave-266d krb5kdc[26158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228531, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:35:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:35:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.87.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
I20260808 22:35:31.477674 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:33889
--webserver_interface=127.31.87.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:40667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:33889
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:35:31.583724 26186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:35:31.583957 26186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:35:31.583995 26186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:35:31.588003 26186 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:35:31.588073 26186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:35:31.588145 26186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:35:31.588181 26186 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:35:31.588210 26186 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:35:31.593372 26186 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:40667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:33889
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:33889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.87.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:35:31.594229 26186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:35:31.595113 26186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:35:31.603963 26196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:31.604056 26199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:31.604149 26186 server_base.cc:1034] running on GCE node
W20260808 22:35:31.604056 26195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:31.604739 26186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:35:31.605824 26186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:35:31.606973 26186 hybrid_clock.cc:648] HybridClock initialized: now 1786228531606961 us; error 27 us; skew 500 ppm
Aug 08 22:35:31 dist-test-slave-266d krb5kdc[26158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228531, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:35:31.609481 26186 init.cc:377] Logged in from keytab as kudu/127.31.87.190@KRBTEST.COM (short username kudu)
I20260808 22:35:31.610308 26186 webserver.cc:460] Webserver started at http://127.31.87.190:44307/ using document root <none> and password file <none>
I20260808 22:35:31.610818 26186 fs_manager.cc:359] Metadata directory not provided
I20260808 22:35:31.610882 26186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:35:31.611061 26186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:35:31.613042 26186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/master-0/data/instance:
uuid: "0bf0ac568c544ceda429040913ef03fb"
format_stamp: "Formatted at 2026-08-08 22:35:31 on dist-test-slave-266d"
server_key: "ab1f6752cc7e7e1eff090df6ee6d8907"
server_key_iv: "ac062eb1de1367f3d125b35924f8025b"
server_key_version: "encryptionkey@0"
I20260808 22:35:31.613623 26186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/master-0/wal/instance:
uuid: "0bf0ac568c544ceda429040913ef03fb"
format_stamp: "Formatted at 2026-08-08 22:35:31 on dist-test-slave-266d"
server_key: "ab1f6752cc7e7e1eff090df6ee6d8907"
server_key_iv: "ac062eb1de1367f3d125b35924f8025b"
server_key_version: "encryptionkey@0"
I20260808 22:35:31.617018 26186 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 22:35:31.619261 26209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:31.620370 26186 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:35:31.620476 26186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "0bf0ac568c544ceda429040913ef03fb"
format_stamp: "Formatted at 2026-08-08 22:35:31 on dist-test-slave-266d"
server_key: "ab1f6752cc7e7e1eff090df6ee6d8907"
server_key_iv: "ac062eb1de1367f3d125b35924f8025b"
server_key_version: "encryptionkey@0"
I20260808 22:35:31.620563 26186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:35:31.633148 26186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:35:31.635860 26186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:35:31.636014 26186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:35:31.642442 26186 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.190:33889
I20260808 22:35:31.642460 26288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.190:33889 every 8 connection(s)
I20260808 22:35:31.643296 26186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:35:31.645900 26289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:35:31.651032 26289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bf0ac568c544ceda429040913ef03fb: Bootstrap starting.
I20260808 22:35:31.652990 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 26186
I20260808 22:35:31.653126 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/master-0/wal/instance
I20260808 22:35:31.653287 26289 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0bf0ac568c544ceda429040913ef03fb: Neither blocks nor log segments found. Creating new log.
I20260808 22:35:31.653409 32094 external_mini_cluster.cc:1428] Setting key 81354d78e6545434d52327dcc447a32d
I20260808 22:35:31.654023 26289 log.cc:824] T 00000000000000000000000000000000 P 0bf0ac568c544ceda429040913ef03fb: Log is configured to *not* fsync() on all Append() calls
I20260808 22:35:31.655865 26289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bf0ac568c544ceda429040913ef03fb: No bootstrap required, opened a new log
I20260808 22:35:31.659116 26289 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0bf0ac568c544ceda429040913ef03fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bf0ac568c544ceda429040913ef03fb" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 33889 } }
I20260808 22:35:31.659317 26289 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0bf0ac568c544ceda429040913ef03fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:35:31.659373 26289 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0bf0ac568c544ceda429040913ef03fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bf0ac568c544ceda429040913ef03fb, State: Initialized, Role: FOLLOWER
I20260808 22:35:31.659816 26289 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0bf0ac568c544ceda429040913ef03fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bf0ac568c544ceda429040913ef03fb" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 33889 } }
I20260808 22:35:31.659962 26289 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0bf0ac568c544ceda429040913ef03fb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:35:31.660030 26289 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0bf0ac568c544ceda429040913ef03fb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:35:31.660130 26289 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0bf0ac568c544ceda429040913ef03fb [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:35:31.661158 26289 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0bf0ac568c544ceda429040913ef03fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bf0ac568c544ceda429040913ef03fb" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 33889 } }
I20260808 22:35:31.661475 26289 leader_election.cc:302] T 00000000000000000000000000000000 P 0bf0ac568c544ceda429040913ef03fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0bf0ac568c544ceda429040913ef03fb; no voters:
Aug 08 22:35:31 dist-test-slave-266d krb5kdc[26158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228531, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:35:31.661741 26289 leader_election.cc:288] T 00000000000000000000000000000000 P 0bf0ac568c544ceda429040913ef03fb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:35:31.661804 26296 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0bf0ac568c544ceda429040913ef03fb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:35:31.661996 26296 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0bf0ac568c544ceda429040913ef03fb [term 1 LEADER]: Becoming Leader. State: Replica: 0bf0ac568c544ceda429040913ef03fb, State: Running, Role: LEADER
I20260808 22:35:31.662328 26296 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0bf0ac568c544ceda429040913ef03fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bf0ac568c544ceda429040913ef03fb" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 33889 } }
I20260808 22:35:31.662917 26289 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0bf0ac568c544ceda429040913ef03fb [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:35:31.663825 26297 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bf0ac568c544ceda429040913ef03fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0bf0ac568c544ceda429040913ef03fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bf0ac568c544ceda429040913ef03fb" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 33889 } } }
I20260808 22:35:31.663944 26297 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bf0ac568c544ceda429040913ef03fb [sys.catalog]: This master's current role is: LEADER
I20260808 22:35:31.664354 26299 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bf0ac568c544ceda429040913ef03fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0bf0ac568c544ceda429040913ef03fb. Latest consensus state: current_term: 1 leader_uuid: "0bf0ac568c544ceda429040913ef03fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bf0ac568c544ceda429040913ef03fb" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 33889 } } }
I20260808 22:35:31.664525 26299 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bf0ac568c544ceda429040913ef03fb [sys.catalog]: This master's current role is: LEADER
I20260808 22:35:31.665380 26308 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:35:31.667196 26295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:31.654398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43272 (local address 127.31.87.190:33889)
0808 22:35:31.654817 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:31.654835 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:31.654877 (+ 42us) server_negotiation.cc:407] Connection header received
0808 22:35:31.655668 (+ 791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:31.655712 (+ 44us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:31.656096 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:31.656401 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:31.657175 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:31.658121 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:31.658942 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:31.659408 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:31.659549 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:31.662224 (+ 2675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:31.662231 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:31.662248 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:35:31.662278 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:31.664157 (+ 1879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:35:31.664659 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:35:31.664665 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:35:31.664671 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:35:31.664818 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:35:31.665176 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:35:31.665179 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:35:31.665181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:35:31.665762 (+ 581us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:31.665920 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:31.666251 (+ 331us) server_negotiation.cc:299] Negotiation successful
0808 22:35:31.666524 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":106,"threads_started":1}
I20260808 22:35:31.668381 26308 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:35:31.673261 26308 catalog_manager.cc:1347] Generated new cluster ID: 6ac7d2770d10414d87a25123c42a4a87
I20260808 22:35:31.673331 26308 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:35:31.699637 26308 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:35:31.700749 26308 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:35:31.705813 26308 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0bf0ac568c544ceda429040913ef03fb: Generated new TSK 0
I20260808 22:35:31.706498 26308 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.87.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab.
Entry for principal kudu/127.31.87.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab.
I20260808 22:35:31.739663 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.129:0
--local_ip_for_outbound_sockets=127.31.87.129
--webserver_interface=127.31.87.129
--webserver_port=0
--tserver_master_addrs=127.31.87.190:33889
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab
--principal=kudu/127.31.87.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:40667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:35:31.848927 26329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:35:31.849196 26329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:35:31.849252 26329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:35:31.854295 26329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:35:31.854360 26329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:35:31.854437 26329 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.129
I20260808 22:35:31.861353 26329 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:40667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab
--principal=kudu/127.31.87.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.87.129
--webserver_port=0
--tserver_master_addrs=127.31.87.190:33889
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26329
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.129
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:35:31.862530 26329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:35:31.863581 26329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:35:31.872886 26337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:31.872923 26335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:31.872969 26334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:31.873211 26329 server_base.cc:1034] running on GCE node
I20260808 22:35:31.873571 26329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:35:31.874157 26329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:35:31.875303 26329 hybrid_clock.cc:648] HybridClock initialized: now 1786228531875280 us; error 38 us; skew 500 ppm
Aug 08 22:35:31 dist-test-slave-266d krb5kdc[26158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228531, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:35:31.877933 26329 init.cc:377] Logged in from keytab as kudu/127.31.87.129@KRBTEST.COM (short username kudu)
I20260808 22:35:31.878687 26329 webserver.cc:460] Webserver started at http://127.31.87.129:46151/ using document root <none> and password file <none>
I20260808 22:35:31.879194 26329 fs_manager.cc:359] Metadata directory not provided
I20260808 22:35:31.879257 26329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:35:31.879464 26329 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:35:31.881413 26329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/ts-0/data/instance:
uuid: "57e260384d0b4716b2576fb6d0f94754"
format_stamp: "Formatted at 2026-08-08 22:35:31 on dist-test-slave-266d"
server_key: "98dc627794f3b4c7a91e64f92fe36698"
server_key_iv: "cf70fe057a7664ceb77a4aa9e29396b9"
server_key_version: "encryptionkey@0"
I20260808 22:35:31.881923 26329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/ts-0/wal/instance:
uuid: "57e260384d0b4716b2576fb6d0f94754"
format_stamp: "Formatted at 2026-08-08 22:35:31 on dist-test-slave-266d"
server_key: "98dc627794f3b4c7a91e64f92fe36698"
server_key_iv: "cf70fe057a7664ceb77a4aa9e29396b9"
server_key_version: "encryptionkey@0"
I20260808 22:35:31.885296 26329 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:35:31.887461 26354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:31.888631 26329 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:35:31.888729 26329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/ts-0/wal
uuid: "57e260384d0b4716b2576fb6d0f94754"
format_stamp: "Formatted at 2026-08-08 22:35:31 on dist-test-slave-266d"
server_key: "98dc627794f3b4c7a91e64f92fe36698"
server_key_iv: "cf70fe057a7664ceb77a4aa9e29396b9"
server_key_version: "encryptionkey@0"
I20260808 22:35:31.888816 26329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:35:31.905045 26329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:35:31.907754 26329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:35:31.907892 26329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:35:31.908381 26329 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:35:31.909135 26329 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:35:31.909183 26329 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:31.909221 26329 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:35:31.909241 26329 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:31.917188 26329 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.129:35455
I20260808 22:35:31.917208 26524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.129:35455 every 8 connection(s)
I20260808 22:35:31.917968 26329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
I20260808 22:35:31.919803 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 26329
I20260808 22:35:31.919932 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/ts-0/wal/instance
I20260808 22:35:31.920274 32094 external_mini_cluster.cc:1428] Setting key b2f6485dbed99eed83344ed305c94cb2
Aug 08 22:35:31 dist-test-slave-266d krb5kdc[26158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228531, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.129@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.87.130@KRBTEST.COM; defaulting to no policy
I20260808 22:35:31.931945 26295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:31.919562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:59041 (local address 127.31.87.190:33889)
0808 22:35:31.919675 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:31.919678 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:31.920565 (+ 887us) server_negotiation.cc:407] Connection header received
0808 22:35:31.922097 (+ 1532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:31.922101 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:31.922155 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:31.922226 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:31.923336 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:31.924100 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:31.925474 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:31.925695 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:31.925775 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:31.928644 (+ 2869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:31.928647 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:31.928649 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:35:31.928672 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:31.929999 (+ 1327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:35:31.930579 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:35:31.930581 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:35:31.930583 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:35:31.930650 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:35:31.931006 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:35:31.931009 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:35:31.931010 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:35:31.931179 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:31.931253 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:31.931688 (+ 435us) server_negotiation.cc:299] Negotiation successful
0808 22:35:31.931805 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 22:35:31.932724 26528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:31.919945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:33889 (local address 127.31.87.129:59041)
0808 22:35:31.920428 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:31.920471 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 22:35:31.921907 (+ 1436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:35:31.922388 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:35:31.922395 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:35:31.922710 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:35:31.923162 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:31.923175 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:31.924223 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:31.924226 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:31.925347 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:31.925359 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:31.925838 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:31.925842 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:31.926026 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:31.926563 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:35:31.926583 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:35:31.928455 (+ 1872us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:35:31.930118 (+ 1663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:35:31.930124 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:35:31.930140 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:35:31.930489 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:35:31.930738 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:35:31.930741 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:35:31.930743 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:35:31.930917 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:35:31.931287 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:35:31.931292 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:35:31.931509 (+ 217us) client_negotiation.cc:769] Sending connection context
0808 22:35:31.931659 (+ 150us) client_negotiation.cc:240] Negotiation successful
0808 22:35:31.931892 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":132,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.130@KRBTEST.COM" created.
I20260808 22:35:31.933971 26525 heartbeater.cc:347] Connected to a master server at 127.31.87.190:33889
I20260808 22:35:31.934219 26525 heartbeater.cc:464] Registering TS with master...
I20260808 22:35:31.934770 26525 heartbeater.cc:511] Master 127.31.87.190:33889 requested a full tablet report, sending...
I20260808 22:35:31.936457 26237 ts_manager.cc:194] Registered new tserver with Master: 57e260384d0b4716b2576fb6d0f94754 (127.31.87.129:35455)
I20260808 22:35:31.937604 26237 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.129@KRBTEST.COM'} at 127.31.87.129:59041
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab.
Entry for principal kudu/127.31.87.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab.
I20260808 22:35:31.949911 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.130:0
--local_ip_for_outbound_sockets=127.31.87.130
--webserver_interface=127.31.87.130
--webserver_port=0
--tserver_master_addrs=127.31.87.190:33889
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab
--principal=kudu/127.31.87.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:40667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:35:32.056188 26531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:35:32.056430 26531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:35:32.056473 26531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:35:32.060326 26531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:35:32.060376 26531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:35:32.060431 26531 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.130
I20260808 22:35:32.065560 26531 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:40667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab
--principal=kudu/127.31.87.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.87.130
--webserver_port=0
--tserver_master_addrs=127.31.87.190:33889
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26531
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.130
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:35:32.066473 26531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:35:32.067405 26531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:35:32.076534 26553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:32.076547 26550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:32.076547 26545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:32.076705 26531 server_base.cc:1034] running on GCE node
I20260808 22:35:32.077118 26531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:35:32.077709 26531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:35:32.078860 26531 hybrid_clock.cc:648] HybridClock initialized: now 1786228532078836 us; error 36 us; skew 500 ppm
Aug 08 22:35:32 dist-test-slave-266d krb5kdc[26158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228532, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:35:32.081408 26531 init.cc:377] Logged in from keytab as kudu/127.31.87.130@KRBTEST.COM (short username kudu)
I20260808 22:35:32.082166 26531 webserver.cc:460] Webserver started at http://127.31.87.130:40711/ using document root <none> and password file <none>
I20260808 22:35:32.082653 26531 fs_manager.cc:359] Metadata directory not provided
I20260808 22:35:32.082710 26531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:35:32.082875 26531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:35:32.084731 26531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/ts-1/data/instance:
uuid: "bbce3924a2ac4009be50cd2b96282c14"
format_stamp: "Formatted at 2026-08-08 22:35:32 on dist-test-slave-266d"
server_key: "d5af6f97c2681d9d4a6b0eff1fa9e5f6"
server_key_iv: "babbb5dd4804844488b4532459e67577"
server_key_version: "encryptionkey@0"
I20260808 22:35:32.085248 26531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/ts-1/wal/instance:
uuid: "bbce3924a2ac4009be50cd2b96282c14"
format_stamp: "Formatted at 2026-08-08 22:35:32 on dist-test-slave-266d"
server_key: "d5af6f97c2681d9d4a6b0eff1fa9e5f6"
server_key_iv: "babbb5dd4804844488b4532459e67577"
server_key_version: "encryptionkey@0"
I20260808 22:35:32.088543 26531 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 22:35:32.090724 26562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:32.091830 26531 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260808 22:35:32.091933 26531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/ts-1/wal
uuid: "bbce3924a2ac4009be50cd2b96282c14"
format_stamp: "Formatted at 2026-08-08 22:35:32 on dist-test-slave-266d"
server_key: "d5af6f97c2681d9d4a6b0eff1fa9e5f6"
server_key_iv: "babbb5dd4804844488b4532459e67577"
server_key_version: "encryptionkey@0"
I20260808 22:35:32.092020 26531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:35:32.114050 26531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:35:32.116948 26531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:35:32.117151 26531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:35:32.117695 26531 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:35:32.118471 26531 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:35:32.118520 26531 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:32.118569 26531 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:35:32.118609 26531 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:32.126968 26531 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.130:46541
I20260808 22:35:32.126994 26718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.130:46541 every 8 connection(s)
I20260808 22:35:32.127790 26531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
I20260808 22:35:32.135494 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 26531
I20260808 22:35:32.135643 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/ts-1/wal/instance
I20260808 22:35:32.135982 32094 external_mini_cluster.cc:1428] Setting key ff8545bde84237b7604124d53583cfdc
Aug 08 22:35:32 dist-test-slave-266d krb5kdc[26158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228532, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.130@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:35:32.140674 26295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:32.129472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:48147 (local address 127.31.87.190:33889)
0808 22:35:32.129602 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:32.129606 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:32.130519 (+ 913us) server_negotiation.cc:407] Connection header received
0808 22:35:32.131698 (+ 1179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:32.131702 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:32.131748 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:32.131818 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:32.132834 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:32.133404 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:32.134422 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:32.134625 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:32.134693 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:32.137228 (+ 2535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:32.137233 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:32.137235 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:35:32.137261 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:32.138758 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:35:32.139205 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:35:32.139209 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:35:32.139210 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:35:32.139304 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:35:32.139706 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:35:32.139708 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:35:32.139709 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:35:32.139876 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:32.139949 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:32.140472 (+ 523us) server_negotiation.cc:299] Negotiation successful
0808 22:35:32.140563 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 22:35:32.141557 26722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:32.129928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:33889 (local address 127.31.87.130:48147)
0808 22:35:32.130377 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:32.130417 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 22:35:32.131518 (+ 1101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:35:32.131956 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:35:32.131962 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:35:32.132288 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:35:32.132726 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:32.132739 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:32.133495 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:32.133498 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:32.134334 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:32.134342 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:32.134724 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:32.134727 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:32.134857 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:32.135351 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:35:32.135371 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:35:32.137044 (+ 1673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:35:32.138863 (+ 1819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:35:32.138867 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:35:32.138878 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:35:32.139120 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:35:32.139407 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:35:32.139410 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:35:32.139412 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:35:32.139591 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:35:32.139964 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:35:32.139970 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:35:32.140254 (+ 284us) client_negotiation.cc:769] Sending connection context
0808 22:35:32.140451 (+ 197us) client_negotiation.cc:240] Negotiation successful
0808 22:35:32.140778 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":116,"threads_started":1}
I20260808 22:35:32.142587 26719 heartbeater.cc:347] Connected to a master server at 127.31.87.190:33889
I20260808 22:35:32.142813 26719 heartbeater.cc:464] Registering TS with master...
I20260808 22:35:32.143348 26719 heartbeater.cc:511] Master 127.31.87.190:33889 requested a full tablet report, sending...
I20260808 22:35:32.144454 26237 ts_manager.cc:194] Registered new tserver with Master: bbce3924a2ac4009be50cd2b96282c14 (127.31.87.130:46541)
I20260808 22:35:32.145233 26237 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.130@KRBTEST.COM'} at 127.31.87.130:48147
WARNING: no policy specified for kudu/127.31.87.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab.
Entry for principal kudu/127.31.87.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab.
I20260808 22:35:32.162055 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.131:0
--local_ip_for_outbound_sockets=127.31.87.131
--webserver_interface=127.31.87.131
--webserver_port=0
--tserver_master_addrs=127.31.87.190:33889
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab
--principal=kudu/127.31.87.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:40667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:35:32.271250 26725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:35:32.271487 26725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:35:32.271553 26725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:35:32.275450 26725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:35:32.275508 26725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:35:32.275600 26725 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.131
I20260808 22:35:32.280653 26725 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:40667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab
--principal=kudu/127.31.87.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.87.131
--webserver_port=0
--tserver_master_addrs=127.31.87.190:33889
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.131
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:35:32.281633 26725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:35:32.282589 26725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:35:32.291816 26732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:32.291870 26738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:32.291872 26731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:32.291996 26725 server_base.cc:1034] running on GCE node
I20260808 22:35:32.292443 26725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:35:32.293033 26725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:35:32.294185 26725 hybrid_clock.cc:648] HybridClock initialized: now 1786228532294170 us; error 29 us; skew 500 ppm
Aug 08 22:35:32 dist-test-slave-266d krb5kdc[26158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228532, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:35:32.296921 26725 init.cc:377] Logged in from keytab as kudu/127.31.87.131@KRBTEST.COM (short username kudu)
I20260808 22:35:32.297995 26725 webserver.cc:460] Webserver started at http://127.31.87.131:36849/ using document root <none> and password file <none>
I20260808 22:35:32.298525 26725 fs_manager.cc:359] Metadata directory not provided
I20260808 22:35:32.298595 26725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:35:32.298802 26725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:35:32.300834 26725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/ts-2/data/instance:
uuid: "cb3825694d724605a9e17d9e664598fa"
format_stamp: "Formatted at 2026-08-08 22:35:32 on dist-test-slave-266d"
server_key: "f615b4781270cd115f5deb6e9cec6080"
server_key_iv: "ad49fef7ab8cf3759daee6c51984847c"
server_key_version: "encryptionkey@0"
I20260808 22:35:32.301422 26725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/ts-2/wal/instance:
uuid: "cb3825694d724605a9e17d9e664598fa"
format_stamp: "Formatted at 2026-08-08 22:35:32 on dist-test-slave-266d"
server_key: "f615b4781270cd115f5deb6e9cec6080"
server_key_iv: "ad49fef7ab8cf3759daee6c51984847c"
server_key_version: "encryptionkey@0"
I20260808 22:35:32.304919 26725 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 22:35:32.307225 26745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:32.308379 26725 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:35:32.308501 26725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/ts-2/wal
uuid: "cb3825694d724605a9e17d9e664598fa"
format_stamp: "Formatted at 2026-08-08 22:35:32 on dist-test-slave-266d"
server_key: "f615b4781270cd115f5deb6e9cec6080"
server_key_iv: "ad49fef7ab8cf3759daee6c51984847c"
server_key_version: "encryptionkey@0"
I20260808 22:35:32.308597 26725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:35:32.328897 26725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:35:32.331948 26725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:35:32.332157 26725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:35:32.332701 26725 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:35:32.333549 26725 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:35:32.333604 26725 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:32.333662 26725 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:35:32.333694 26725 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:32.342770 26725 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.131:42281
I20260808 22:35:32.342788 26918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.131:42281 every 8 connection(s)
I20260808 22:35:32.343655 26725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
I20260808 22:35:32.347827 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 26725
I20260808 22:35:32.347935 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/ts-2/wal/instance
I20260808 22:35:32.348203 32094 external_mini_cluster.cc:1428] Setting key dc3f9e52385ae73b7577c144b6c64aaa
Aug 08 22:35:32 dist-test-slave-266d krb5kdc[26158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228532, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.131@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:35:32.356693 26295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:32.345326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:47335 (local address 127.31.87.190:33889)
0808 22:35:32.345462 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:32.345466 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:32.346248 (+ 782us) server_negotiation.cc:407] Connection header received
0808 22:35:32.347514 (+ 1266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:32.347518 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:32.347605 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:32.347689 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:32.348906 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:32.349502 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:32.350659 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:32.350881 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:32.350970 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:32.353410 (+ 2440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:32.353414 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:32.353416 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:35:32.353440 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:32.354640 (+ 1200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:35:32.355151 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:35:32.355155 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:35:32.355156 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:35:32.355242 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:35:32.355580 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:35:32.355583 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:35:32.355585 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:35:32.355776 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:32.355875 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:32.356374 (+ 499us) server_negotiation.cc:299] Negotiation successful
0808 22:35:32.356508 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 22:35:32.357425 26927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:32.345650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:33889 (local address 127.31.87.131:47335)
0808 22:35:32.346093 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:32.346134 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 22:35:32.347298 (+ 1164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:35:32.347866 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:35:32.347876 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:35:32.348276 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:35:32.348748 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:32.348763 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:32.349630 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:32.349635 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:32.350520 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:32.350531 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:32.351033 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:32.351038 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:32.351207 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:32.351755 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:35:32.351772 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:35:32.353244 (+ 1472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:35:32.354756 (+ 1512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:35:32.354763 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:35:32.354778 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:35:32.355033 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:35:32.355330 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:35:32.355333 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:35:32.355335 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:35:32.355474 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:35:32.355883 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:35:32.355890 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:35:32.356127 (+ 237us) client_negotiation.cc:769] Sending connection context
0808 22:35:32.356300 (+ 173us) client_negotiation.cc:240] Negotiation successful
0808 22:35:32.356622 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":111,"threads_started":1}
I20260808 22:35:32.358479 26925 heartbeater.cc:347] Connected to a master server at 127.31.87.190:33889
I20260808 22:35:32.358716 26925 heartbeater.cc:464] Registering TS with master...
I20260808 22:35:32.359153 26925 heartbeater.cc:511] Master 127.31.87.190:33889 requested a full tablet report, sending...
I20260808 22:35:32.360102 26237 ts_manager.cc:194] Registered new tserver with Master: cb3825694d724605a9e17d9e664598fa (127.31.87.131:42281)
I20260808 22:35:32.360682 26237 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.131@KRBTEST.COM'} at 127.31.87.131:47335
I20260808 22:35:32.361697 32094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:35:32.370690 26295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:32.363650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43274 (local address 127.31.87.190:33889)
0808 22:35:32.363763 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:32.363768 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:32.363948 (+ 180us) server_negotiation.cc:407] Connection header received
0808 22:35:32.364072 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:32.364075 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:32.364119 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:32.364185 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:32.364940 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:32.365496 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:32.366335 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:32.366565 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:32.366656 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:32.367894 (+ 1238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:32.367899 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:32.367902 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:35:32.367929 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:32.369016 (+ 1087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:35:32.369457 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:35:32.369461 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:35:32.369463 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:35:32.369547 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:35:32.369823 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:35:32.369827 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:35:32.369829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:35:32.370045 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:32.370175 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:32.370413 (+ 238us) server_negotiation.cc:299] Negotiation successful
0808 22:35:32.370521 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 22:35:32.373054 26237 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43274:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:35:32.375818 26237 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:35:32.389391 26948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:32.384094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38650 (local address 127.31.87.130:46541)
0808 22:35:32.384717 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:32.384728 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:32.384754 (+ 26us) server_negotiation.cc:407] Connection header received
0808 22:35:32.384924 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:32.384931 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:32.385163 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:32.385346 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:35:32.385615 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:32.386785 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:32.388090 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:32.388742 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:32.388841 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:32.388985 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:32.389157 (+ 172us) server_negotiation.cc:299] Negotiation successful
0808 22:35:32.389255 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":99,"threads_started":1}
I20260808 22:35:32.389384 26943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:32.383640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.130:46541 (local address 127.0.0.1:38650)
0808 22:35:32.384672 (+ 1032us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:32.384685 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:35:32.384755 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:35:32.385335 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:35:32.385341 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:35:32.385364 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:35:32.385492 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:32.385499 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:32.386918 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:32.386922 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:32.387986 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:32.387994 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:32.388852 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:32.388856 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:32.388968 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:32.389005 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 22:35:32.389125 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 22:35:32.389248 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":973,"mutex_wait_us":1,"thread_start_us":88,"threads_started":1}
I20260808 22:35:32.389683 26947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:32.383645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47022 (local address 127.31.87.131:42281)
0808 22:35:32.383926 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:32.383937 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:32.384422 (+ 485us) server_negotiation.cc:407] Connection header received
0808 22:35:32.385073 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:32.385078 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:32.385253 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:32.385371 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:35:32.385780 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:32.386466 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:32.387935 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:32.388754 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:32.388853 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:32.388984 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:32.389446 (+ 462us) server_negotiation.cc:299] Negotiation successful
0808 22:35:32.389537 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":117,"threads_started":1}
I20260808 22:35:32.389690 26945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:32.383519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.131:42281 (local address 127.0.0.1:47022)
0808 22:35:32.384291 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:32.384337 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 22:35:32.384468 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:35:32.385497 (+ 1029us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:35:32.385500 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:35:32.385517 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:35:32.385639 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:32.385647 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:32.386585 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:32.386588 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:32.387826 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:32.387837 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:32.389048 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:32.389053 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:32.389167 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:32.389192 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 22:35:32.389253 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 22:35:32.389320 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"mutex_wait_us":18,"thread_start_us":63,"threads_started":1}
I20260808 22:35:32.389912 26949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:32.384203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:35455 (local address 127.0.0.1:52370)
0808 22:35:32.384761 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:32.384775 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:35:32.384866 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:35:32.385509 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:35:32.385512 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:35:32.385525 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:35:32.385664 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:32.385670 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:32.387135 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:32.387138 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:32.388223 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:32.388233 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:32.389430 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:32.389433 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:32.389522 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:32.389544 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 22:35:32.389609 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 22:35:32.389669 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"mutex_wait_us":38,"spinlock_wait_cycles":384,"thread_start_us":104,"threads_started":1}
I20260808 22:35:32.390177 26951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:32.384367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52370 (local address 127.31.87.129:35455)
0808 22:35:32.384947 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:32.384958 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:32.384979 (+ 21us) server_negotiation.cc:407] Connection header received
0808 22:35:32.385050 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:32.385055 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:32.385262 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:32.385406 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:35:32.385770 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:32.386874 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:32.388355 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:32.389153 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:32.389250 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:32.389369 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:32.389758 (+ 389us) server_negotiation.cc:299] Negotiation successful
0808 22:35:32.389955 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":124,"threads_started":1}
I20260808 22:35:32.392400 26817 tablet_service.cc:1467] Processing CreateTablet for tablet 9d8db2db94384150affc01c7c99322d1 (DEFAULT_TABLE table=test-table [id=8b56d48f2fa24db5b8f22bb2366d07fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:35:32.392705 26433 tablet_service.cc:1467] Processing CreateTablet for tablet 9d8db2db94384150affc01c7c99322d1 (DEFAULT_TABLE table=test-table [id=8b56d48f2fa24db5b8f22bb2366d07fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:35:32.393028 26631 tablet_service.cc:1467] Processing CreateTablet for tablet 9d8db2db94384150affc01c7c99322d1 (DEFAULT_TABLE table=test-table [id=8b56d48f2fa24db5b8f22bb2366d07fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:35:32.393429 26817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d8db2db94384150affc01c7c99322d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:35:32.393745 26433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d8db2db94384150affc01c7c99322d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:35:32.393946 26631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d8db2db94384150affc01c7c99322d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:35:32.399236 26952 tablet_bootstrap.cc:492] T 9d8db2db94384150affc01c7c99322d1 P cb3825694d724605a9e17d9e664598fa: Bootstrap starting.
I20260808 22:35:32.400921 26954 tablet_bootstrap.cc:492] T 9d8db2db94384150affc01c7c99322d1 P 57e260384d0b4716b2576fb6d0f94754: Bootstrap starting.
I20260808 22:35:32.401371 26955 tablet_bootstrap.cc:492] T 9d8db2db94384150affc01c7c99322d1 P bbce3924a2ac4009be50cd2b96282c14: Bootstrap starting.
I20260808 22:35:32.401942 26952 tablet_bootstrap.cc:654] T 9d8db2db94384150affc01c7c99322d1 P cb3825694d724605a9e17d9e664598fa: Neither blocks nor log segments found. Creating new log.
I20260808 22:35:32.402988 26955 tablet_bootstrap.cc:654] T 9d8db2db94384150affc01c7c99322d1 P bbce3924a2ac4009be50cd2b96282c14: Neither blocks nor log segments found. Creating new log.
I20260808 22:35:32.403048 26954 tablet_bootstrap.cc:654] T 9d8db2db94384150affc01c7c99322d1 P 57e260384d0b4716b2576fb6d0f94754: Neither blocks nor log segments found. Creating new log.
I20260808 22:35:32.403218 26952 log.cc:824] T 9d8db2db94384150affc01c7c99322d1 P cb3825694d724605a9e17d9e664598fa: Log is configured to *not* fsync() on all Append() calls
I20260808 22:35:32.404043 26955 log.cc:824] T 9d8db2db94384150affc01c7c99322d1 P bbce3924a2ac4009be50cd2b96282c14: Log is configured to *not* fsync() on all Append() calls
I20260808 22:35:32.404238 26954 log.cc:824] T 9d8db2db94384150affc01c7c99322d1 P 57e260384d0b4716b2576fb6d0f94754: Log is configured to *not* fsync() on all Append() calls
I20260808 22:35:32.405615 26952 tablet_bootstrap.cc:492] T 9d8db2db94384150affc01c7c99322d1 P cb3825694d724605a9e17d9e664598fa: No bootstrap required, opened a new log
I20260808 22:35:32.405812 26952 ts_tablet_manager.cc:1397] T 9d8db2db94384150affc01c7c99322d1 P cb3825694d724605a9e17d9e664598fa: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 22:35:32.405853 26954 tablet_bootstrap.cc:492] T 9d8db2db94384150affc01c7c99322d1 P 57e260384d0b4716b2576fb6d0f94754: No bootstrap required, opened a new log
I20260808 22:35:32.405907 26955 tablet_bootstrap.cc:492] T 9d8db2db94384150affc01c7c99322d1 P bbce3924a2ac4009be50cd2b96282c14: No bootstrap required, opened a new log
I20260808 22:35:32.406021 26954 ts_tablet_manager.cc:1397] T 9d8db2db94384150affc01c7c99322d1 P 57e260384d0b4716b2576fb6d0f94754: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 22:35:32.406075 26955 ts_tablet_manager.cc:1397] T 9d8db2db94384150affc01c7c99322d1 P bbce3924a2ac4009be50cd2b96282c14: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 22:35:32.408674 26954 raft_consensus.cc:348] T 9d8db2db94384150affc01c7c99322d1 P 57e260384d0b4716b2576fb6d0f94754 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3825694d724605a9e17d9e664598fa" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 42281 } } peers { permanent_uuid: "bbce3924a2ac4009be50cd2b96282c14" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 46541 } } peers { permanent_uuid: "57e260384d0b4716b2576fb6d0f94754" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 35455 } }
I20260808 22:35:32.408943 26954 raft_consensus.cc:374] T 9d8db2db94384150affc01c7c99322d1 P 57e260384d0b4716b2576fb6d0f94754 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:35:32.409010 26954 raft_consensus.cc:729] T 9d8db2db94384150affc01c7c99322d1 P 57e260384d0b4716b2576fb6d0f94754 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e260384d0b4716b2576fb6d0f94754, State: Initialized, Role: FOLLOWER
I20260808 22:35:32.409457 26954 consensus_queue.cc:260] T 9d8db2db94384150affc01c7c99322d1 P 57e260384d0b4716b2576fb6d0f94754 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3825694d724605a9e17d9e664598fa" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 42281 } } peers { permanent_uuid: "bbce3924a2ac4009be50cd2b96282c14" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 46541 } } peers { permanent_uuid: "57e260384d0b4716b2576fb6d0f94754" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 35455 } }
I20260808 22:35:32.409569 26952 raft_consensus.cc:348] T 9d8db2db94384150affc01c7c99322d1 P cb3825694d724605a9e17d9e664598fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3825694d724605a9e17d9e664598fa" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 42281 } } peers { permanent_uuid: "bbce3924a2ac4009be50cd2b96282c14" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 46541 } } peers { permanent_uuid: "57e260384d0b4716b2576fb6d0f94754" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 35455 } }
I20260808 22:35:32.409569 26955 raft_consensus.cc:348] T 9d8db2db94384150affc01c7c99322d1 P bbce3924a2ac4009be50cd2b96282c14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3825694d724605a9e17d9e664598fa" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 42281 } } peers { permanent_uuid: "bbce3924a2ac4009be50cd2b96282c14" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 46541 } } peers { permanent_uuid: "57e260384d0b4716b2576fb6d0f94754" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 35455 } }
I20260808 22:35:32.409813 26955 raft_consensus.cc:374] T 9d8db2db94384150affc01c7c99322d1 P bbce3924a2ac4009be50cd2b96282c14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:35:32.409813 26952 raft_consensus.cc:374] T 9d8db2db94384150affc01c7c99322d1 P cb3825694d724605a9e17d9e664598fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:35:32.409873 26955 raft_consensus.cc:729] T 9d8db2db94384150affc01c7c99322d1 P bbce3924a2ac4009be50cd2b96282c14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbce3924a2ac4009be50cd2b96282c14, State: Initialized, Role: FOLLOWER
I20260808 22:35:32.409888 26952 raft_consensus.cc:729] T 9d8db2db94384150affc01c7c99322d1 P cb3825694d724605a9e17d9e664598fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb3825694d724605a9e17d9e664598fa, State: Initialized, Role: FOLLOWER
I20260808 22:35:32.410202 26525 heartbeater.cc:503] Master 127.31.87.190:33889 was elected leader, sending a full tablet report...
I20260808 22:35:32.410291 26955 consensus_queue.cc:260] T 9d8db2db94384150affc01c7c99322d1 P bbce3924a2ac4009be50cd2b96282c14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3825694d724605a9e17d9e664598fa" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 42281 } } peers { permanent_uuid: "bbce3924a2ac4009be50cd2b96282c14" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 46541 } } peers { permanent_uuid: "57e260384d0b4716b2576fb6d0f94754" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 35455 } }
I20260808 22:35:32.410313 26952 consensus_queue.cc:260] T 9d8db2db94384150affc01c7c99322d1 P cb3825694d724605a9e17d9e664598fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3825694d724605a9e17d9e664598fa" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 42281 } } peers { permanent_uuid: "bbce3924a2ac4009be50cd2b96282c14" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 46541 } } peers { permanent_uuid: "57e260384d0b4716b2576fb6d0f94754" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 35455 } }
I20260808 22:35:32.410409 26954 ts_tablet_manager.cc:1428] T 9d8db2db94384150affc01c7c99322d1 P 57e260384d0b4716b2576fb6d0f94754: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260808 22:35:32.411192 26955 ts_tablet_manager.cc:1428] T 9d8db2db94384150affc01c7c99322d1 P bbce3924a2ac4009be50cd2b96282c14: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 22:35:32.411192 26952 ts_tablet_manager.cc:1428] T 9d8db2db94384150affc01c7c99322d1 P cb3825694d724605a9e17d9e664598fa: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 22:35:32.411540 26719 heartbeater.cc:503] Master 127.31.87.190:33889 was elected leader, sending a full tablet report...
I20260808 22:35:32.411823 26925 heartbeater.cc:503] Master 127.31.87.190:33889 was elected leader, sending a full tablet report...
W20260808 22:35:32.418951 26527 tablet.cc:2369] T 9d8db2db94384150affc01c7c99322d1 P 57e260384d0b4716b2576fb6d0f94754: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:35:32.589468 26959 raft_consensus.cc:482] T 9d8db2db94384150affc01c7c99322d1 P 57e260384d0b4716b2576fb6d0f94754 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:35:32.589669 26959 raft_consensus.cc:504] T 9d8db2db94384150affc01c7c99322d1 P 57e260384d0b4716b2576fb6d0f94754 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3825694d724605a9e17d9e664598fa" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 42281 } } peers { permanent_uuid: "bbce3924a2ac4009be50cd2b96282c14" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 46541 } } peers { permanent_uuid: "57e260384d0b4716b2576fb6d0f94754" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 35455 } }
I20260808 22:35:32.590750 26959 leader_election.cc:288] T 9d8db2db94384150affc01c7c99322d1 P 57e260384d0b4716b2576fb6d0f94754 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb3825694d724605a9e17d9e664598fa (127.31.87.131:42281), bbce3924a2ac4009be50cd2b96282c14 (127.31.87.130:46541)
W20260808 22:35:32.594525 26926 tablet.cc:2369] T 9d8db2db94384150affc01c7c99322d1 P cb3825694d724605a9e17d9e664598fa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:35:32.595598 26947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:32.591196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:37091 (local address 127.31.87.131:42281)
0808 22:35:32.591305 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:32.591325 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:32.591344 (+ 19us) server_negotiation.cc:407] Connection header received
0808 22:35:32.591502 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:32.591505 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:32.591548 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:32.591616 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:35:32.591968 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:32.592704 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:32.594344 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:32.594942 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:32.595076 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:32.595126 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:32.595403 (+ 277us) server_negotiation.cc:299] Negotiation successful
0808 22:35:32.595471 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 22:35:32.595598 26948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:32.591102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:44697 (local address 127.31.87.130:46541)
0808 22:35:32.591306 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:32.591310 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:32.591651 (+ 341us) server_negotiation.cc:407] Connection header received
0808 22:35:32.591794 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:32.591797 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:32.591840 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:32.591901 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:35:32.592166 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:32.592917 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:32.594344 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:32.594942 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:32.595070 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:32.595125 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:32.595402 (+ 277us) server_negotiation.cc:299] Negotiation successful
0808 22:35:32.595471 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260808 22:35:32.595667 26964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:32.590973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.130:46541 (local address 127.31.87.129:44697)
0808 22:35:32.591459 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:32.591471 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:35:32.591694 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:35:32.591930 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:35:32.591933 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:35:32.591952 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:35:32.592067 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:32.592073 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:32.593032 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:32.593035 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:32.594249 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:32.594257 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:32.595112 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:32.595115 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:32.595273 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:32.595310 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 22:35:32.595386 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 22:35:32.595487 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427,"mutex_wait_us":97,"thread_start_us":126,"threads_started":1}
I20260808 22:35:32.595993 26963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:32.590973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.131:42281 (local address 127.31.87.129:37091)
0808 22:35:32.591278 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:32.591293 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:35:32.591399 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:35:32.591686 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:35:32.591690 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:35:32.591705 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:35:32.591847 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:32.591854 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:32.592838 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:32.592841 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:32.594207 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:32.594228 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:32.595074 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:32.595078 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:32.595280 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:32.595311 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 22:35:32.595386 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 22:35:32.595487 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"mutex_wait_us":64,"thread_start_us":69,"threads_started":1}
I20260808 22:35:32.596377 26658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d8db2db94384150affc01c7c99322d1" candidate_uuid: "57e260384d0b4716b2576fb6d0f94754" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbce3924a2ac4009be50cd2b96282c14" is_pre_election: true
I20260808 22:35:32.596676 26849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d8db2db94384150affc01c7c99322d1" candidate_uuid: "57e260384d0b4716b2576fb6d0f94754" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb3825694d724605a9e17d9e664598fa" is_pre_election: true
I20260808 22:35:32.596717 26658 raft_consensus.cc:2454] T 9d8db2db94384150affc01c7c99322d1 P bbce3924a2ac4009be50cd2b96282c14 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e260384d0b4716b2576fb6d0f94754 in term 0.
I20260808 22:35:32.596912 26849 raft_consensus.cc:2454] T 9d8db2db94384150affc01c7c99322d1 P cb3825694d724605a9e17d9e664598fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e260384d0b4716b2576fb6d0f94754 in term 0.
I20260808 22:35:32.597254 26355 leader_election.cc:302] T 9d8db2db94384150affc01c7c99322d1 P 57e260384d0b4716b2576fb6d0f94754 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e260384d0b4716b2576fb6d0f94754, bbce3924a2ac4009be50cd2b96282c14; no voters:
I20260808 22:35:32.597514 26959 raft_consensus.cc:2781] T 9d8db2db94384150affc01c7c99322d1 P 57e260384d0b4716b2576fb6d0f94754 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:35:32.597597 26959 raft_consensus.cc:482] T 9d8db2db94384150affc01c7c99322d1 P 57e260384d0b4716b2576fb6d0f94754 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:35:32.597630 26959 raft_consensus.cc:3037] T 9d8db2db94384150affc01c7c99322d1 P 57e260384d0b4716b2576fb6d0f94754 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:35:32.598650 26959 raft_consensus.cc:504] T 9d8db2db94384150affc01c7c99322d1 P 57e260384d0b4716b2576fb6d0f94754 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3825694d724605a9e17d9e664598fa" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 42281 } } peers { permanent_uuid: "bbce3924a2ac4009be50cd2b96282c14" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 46541 } } peers { permanent_uuid: "57e260384d0b4716b2576fb6d0f94754" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 35455 } }
I20260808 22:35:32.598976 26959 leader_election.cc:288] T 9d8db2db94384150affc01c7c99322d1 P 57e260384d0b4716b2576fb6d0f94754 [CANDIDATE]: Term 1 election: Requested vote from peers cb3825694d724605a9e17d9e664598fa (127.31.87.131:42281), bbce3924a2ac4009be50cd2b96282c14 (127.31.87.130:46541)
I20260808 22:35:32.599357 26658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d8db2db94384150affc01c7c99322d1" candidate_uuid: "57e260384d0b4716b2576fb6d0f94754" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbce3924a2ac4009be50cd2b96282c14"
I20260808 22:35:32.599481 26658 raft_consensus.cc:3037] T 9d8db2db94384150affc01c7c99322d1 P bbce3924a2ac4009be50cd2b96282c14 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:35:32.599818 26849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d8db2db94384150affc01c7c99322d1" candidate_uuid: "57e260384d0b4716b2576fb6d0f94754" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb3825694d724605a9e17d9e664598fa"
I20260808 22:35:32.599939 26849 raft_consensus.cc:3037] T 9d8db2db94384150affc01c7c99322d1 P cb3825694d724605a9e17d9e664598fa [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:35:32.600822 26658 raft_consensus.cc:2454] T 9d8db2db94384150affc01c7c99322d1 P bbce3924a2ac4009be50cd2b96282c14 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e260384d0b4716b2576fb6d0f94754 in term 1.
I20260808 22:35:32.601217 26355 leader_election.cc:302] T 9d8db2db94384150affc01c7c99322d1 P 57e260384d0b4716b2576fb6d0f94754 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e260384d0b4716b2576fb6d0f94754, bbce3924a2ac4009be50cd2b96282c14; no voters:
I20260808 22:35:32.601234 26849 raft_consensus.cc:2454] T 9d8db2db94384150affc01c7c99322d1 P cb3825694d724605a9e17d9e664598fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e260384d0b4716b2576fb6d0f94754 in term 1.
I20260808 22:35:32.601475 26959 raft_consensus.cc:2781] T 9d8db2db94384150affc01c7c99322d1 P 57e260384d0b4716b2576fb6d0f94754 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:35:32.601779 26959 raft_consensus.cc:686] T 9d8db2db94384150affc01c7c99322d1 P 57e260384d0b4716b2576fb6d0f94754 [term 1 LEADER]: Becoming Leader. State: Replica: 57e260384d0b4716b2576fb6d0f94754, State: Running, Role: LEADER
I20260808 22:35:32.602139 26959 consensus_queue.cc:237] T 9d8db2db94384150affc01c7c99322d1 P 57e260384d0b4716b2576fb6d0f94754 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3825694d724605a9e17d9e664598fa" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 42281 } } peers { permanent_uuid: "bbce3924a2ac4009be50cd2b96282c14" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 46541 } } peers { permanent_uuid: "57e260384d0b4716b2576fb6d0f94754" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 35455 } }
I20260808 22:35:32.605507 26235 catalog_manager.cc:5495] T 9d8db2db94384150affc01c7c99322d1 P 57e260384d0b4716b2576fb6d0f94754 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e260384d0b4716b2576fb6d0f94754 (127.31.87.129). New cstate: current_term: 1 leader_uuid: "57e260384d0b4716b2576fb6d0f94754" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3825694d724605a9e17d9e664598fa" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 42281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bbce3924a2ac4009be50cd2b96282c14" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 46541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e260384d0b4716b2576fb6d0f94754" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 35455 } health_report { overall_health: HEALTHY } } }
W20260808 22:35:32.628938 26720 tablet.cc:2369] T 9d8db2db94384150affc01c7c99322d1 P bbce3924a2ac4009be50cd2b96282c14: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:35:32.630604 26951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:32.626933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52384 (local address 127.31.87.129:35455)
0808 22:35:32.627056 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:32.627060 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:32.627074 (+ 14us) server_negotiation.cc:407] Connection header received
0808 22:35:32.627228 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:32.627231 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:32.627284 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:32.627371 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:35:32.627606 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:32.628089 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:32.629025 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:32.629292 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:32.629374 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:32.629670 (+ 296us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:35:32.630185 (+ 515us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:35:32.630309 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:32.630401 (+ 92us) server_negotiation.cc:299] Negotiation successful
0808 22:35:32.630466 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 22:35:32.635932 26658 raft_consensus.cc:1261] T 9d8db2db94384150affc01c7c99322d1 P bbce3924a2ac4009be50cd2b96282c14 [term 1 FOLLOWER]: Refusing update from remote peer 57e260384d0b4716b2576fb6d0f94754: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:35:32.635934 26849 raft_consensus.cc:1261] T 9d8db2db94384150affc01c7c99322d1 P cb3825694d724605a9e17d9e664598fa [term 1 FOLLOWER]: Refusing update from remote peer 57e260384d0b4716b2576fb6d0f94754: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:35:32.636593 26965 consensus_queue.cc:1035] T 9d8db2db94384150affc01c7c99322d1 P 57e260384d0b4716b2576fb6d0f94754 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbce3924a2ac4009be50cd2b96282c14" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 46541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:35:32.636766 26959 consensus_queue.cc:1035] T 9d8db2db94384150affc01c7c99322d1 P 57e260384d0b4716b2576fb6d0f94754 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb3825694d724605a9e17d9e664598fa" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 42281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:35:32.644433 26968 mvcc.cc:205] Tried to move back new op lower bound from 7316392069670539264 to 7316392069541101568. Current Snapshot: MvccSnapshot[applied={T|T < 7316392069670539264}]
I20260808 22:35:32.652092 26970 mvcc.cc:205] Tried to move back new op lower bound from 7316392069670539264 to 7316392069541101568. Current Snapshot: MvccSnapshot[applied={T|T < 7316392069670539264}]
I20260808 22:35:32.654222 26236 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43274:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:35:32.654363 26236 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43274:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:35:32.656555 26236 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0bf0ac568c544ceda429040913ef03fb: Sending DeleteTablet for 3 replicas of tablet 9d8db2db94384150affc01c7c99322d1
I20260808 22:35:32.657351 26631 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d8db2db94384150affc01c7c99322d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:35:32 UTC) from {username='kudu', principal='kudu/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:38650
I20260808 22:35:32.657341 26817 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d8db2db94384150affc01c7c99322d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:35:32 UTC) from {username='kudu', principal='kudu/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:47022
I20260808 22:35:32.657778 26982 tablet_replica.cc:330] T 9d8db2db94384150affc01c7c99322d1 P bbce3924a2ac4009be50cd2b96282c14: stopping tablet replica
I20260808 22:35:32.657816 26433 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d8db2db94384150affc01c7c99322d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:35:32 UTC) from {username='kudu', principal='kudu/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:52370
I20260808 22:35:32.658185 26982 raft_consensus.cc:2229] T 9d8db2db94384150affc01c7c99322d1 P bbce3924a2ac4009be50cd2b96282c14 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:35:32.658766 26982 raft_consensus.cc:2258] T 9d8db2db94384150affc01c7c99322d1 P bbce3924a2ac4009be50cd2b96282c14 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:35:32.658892 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 26329
I20260808 22:35:32.668532 26983 tablet_replica.cc:330] T 9d8db2db94384150affc01c7c99322d1 P cb3825694d724605a9e17d9e664598fa: stopping tablet replica
I20260808 22:35:32.668803 26983 raft_consensus.cc:2229] T 9d8db2db94384150affc01c7c99322d1 P cb3825694d724605a9e17d9e664598fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:35:32.669041 26983 raft_consensus.cc:2258] T 9d8db2db94384150affc01c7c99322d1 P cb3825694d724605a9e17d9e664598fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:35:32.670045 26982 ts_tablet_manager.cc:1905] T 9d8db2db94384150affc01c7c99322d1 P bbce3924a2ac4009be50cd2b96282c14: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:35:32.670387 26983 ts_tablet_manager.cc:1905] T 9d8db2db94384150affc01c7c99322d1 P cb3825694d724605a9e17d9e664598fa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:35:32.672200 26982 ts_tablet_manager.cc:1918] T 9d8db2db94384150affc01c7c99322d1 P bbce3924a2ac4009be50cd2b96282c14: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:35:32.672304 26982 log.cc:1196] T 9d8db2db94384150affc01c7c99322d1 P bbce3924a2ac4009be50cd2b96282c14: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/ts-1/wal/wals/9d8db2db94384150affc01c7c99322d1
I20260808 22:35:32.672560 26982 ts_tablet_manager.cc:1939] T 9d8db2db94384150affc01c7c99322d1 P bbce3924a2ac4009be50cd2b96282c14: Deleting consensus metadata
I20260808 22:35:32.672907 26983 ts_tablet_manager.cc:1918] T 9d8db2db94384150affc01c7c99322d1 P cb3825694d724605a9e17d9e664598fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:35:32.672999 26983 log.cc:1196] T 9d8db2db94384150affc01c7c99322d1 P cb3825694d724605a9e17d9e664598fa: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786228291013016-32094-0/minicluster-data/ts-2/wal/wals/9d8db2db94384150affc01c7c99322d1
I20260808 22:35:32.673322 26983 ts_tablet_manager.cc:1939] T 9d8db2db94384150affc01c7c99322d1 P cb3825694d724605a9e17d9e664598fa: Deleting consensus metadata
I20260808 22:35:32.673707 26210 catalog_manager.cc:4841] TS bbce3924a2ac4009be50cd2b96282c14 (127.31.87.130:46541): tablet 9d8db2db94384150affc01c7c99322d1 (table test-table [id=8b56d48f2fa24db5b8f22bb2366d07fb]) successfully deleted
I20260808 22:35:32.674407 26217 catalog_manager.cc:4841] TS cb3825694d724605a9e17d9e664598fa (127.31.87.131:42281): tablet 9d8db2db94384150affc01c7c99322d1 (table test-table [id=8b56d48f2fa24db5b8f22bb2366d07fb]) successfully deleted
W20260808 22:35:32.676457 26217 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.87.129:35455 (error 108)
I20260808 22:35:32.676517 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 26531
I20260808 22:35:32.682824 26943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:32.682567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:35455 (local address 127.0.0.1:52396)
0808 22:35:32.682693 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:32.682754 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.129:35455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260808 22:35:32.682972 26217 catalog_manager.cc:4584] TS 57e260384d0b4716b2576fb6d0f94754 (127.31.87.129:35455): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9d8db2db94384150affc01c7c99322d1: Network error: Client connection negotiation failed: client connection to 127.31.87.129:35455: connect: Connection refused (error 111)
I20260808 22:35:32.686885 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 26725
I20260808 22:35:32.699316 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 26186
2026-08-08T22:35:32Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2472 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-08T22:35:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:35:32Z Disabled control of system clock
I20260808 22:35:32.735448 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:39419
--webserver_interface=127.31.87.190
--webserver_port=0
--builtin_ntp_servers=127.31.87.148:45965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:39419
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 22:35:32.841153 26993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:35:32.841382 26993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:35:32.841413 26993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:35:32.845206 26993 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:35:32.845254 26993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:35:32.845274 26993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:35:32.845295 26993 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:35:32.845312 26993 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:35:32.849818 26993 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:45965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:39419
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:39419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.87.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26993
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:35:32.850695 26993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:35:32.851891 26993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:35:32.859695 27003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:32.859814 26993 server_base.cc:1034] running on GCE node
W20260808 22:35:32.859695 26999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:32.859964 27000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:32.860476 26993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:35:32.861557 26993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:35:32.862717 26993 hybrid_clock.cc:648] HybridClock initialized: now 1786228532862705 us; error 28 us; skew 500 ppm
I20260808 22:35:32.864399 26993 webserver.cc:460] Webserver started at http://127.31.87.190:41209/ using document root <none> and password file <none>
I20260808 22:35:32.864935 26993 fs_manager.cc:359] Metadata directory not provided
I20260808 22:35:32.864995 26993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:35:32.865236 26993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:35:32.867151 26993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/master-0/data/instance:
uuid: "a45697e94a0f43b7ada7975ad01c6930"
format_stamp: "Formatted at 2026-08-08 22:35:32 on dist-test-slave-266d"
server_key: "4cf6c7749b23e713f10e80a2d5dfd6a7"
server_key_iv: "41b3e148c8475d26c8d24c3a17d20d05"
server_key_version: "encryptionkey@0"
I20260808 22:35:32.867677 26993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/master-0/wal/instance:
uuid: "a45697e94a0f43b7ada7975ad01c6930"
format_stamp: "Formatted at 2026-08-08 22:35:32 on dist-test-slave-266d"
server_key: "4cf6c7749b23e713f10e80a2d5dfd6a7"
server_key_iv: "41b3e148c8475d26c8d24c3a17d20d05"
server_key_version: "encryptionkey@0"
I20260808 22:35:32.871156 26993 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 22:35:32.873380 27015 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:32.874532 26993 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:35:32.874650 26993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "a45697e94a0f43b7ada7975ad01c6930"
format_stamp: "Formatted at 2026-08-08 22:35:32 on dist-test-slave-266d"
server_key: "4cf6c7749b23e713f10e80a2d5dfd6a7"
server_key_iv: "41b3e148c8475d26c8d24c3a17d20d05"
server_key_version: "encryptionkey@0"
I20260808 22:35:32.874739 26993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:35:32.892692 26993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:35:32.893348 26993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:35:32.893512 26993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:35:32.900044 26993 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.190:39419
I20260808 22:35:32.900058 27081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.190:39419 every 8 connection(s)
I20260808 22:35:32.900900 26993 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:35:32.903753 27082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:35:32.908860 27082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a45697e94a0f43b7ada7975ad01c6930: Bootstrap starting.
I20260808 22:35:32.910606 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 26993
I20260808 22:35:32.910737 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/master-0/wal/instance
I20260808 22:35:32.911041 32094 external_mini_cluster.cc:1428] Setting key 66dced5eb109cd39db24aa88fff5fc8d
I20260808 22:35:32.911329 27082 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a45697e94a0f43b7ada7975ad01c6930: Neither blocks nor log segments found. Creating new log.
I20260808 22:35:32.912142 27082 log.cc:824] T 00000000000000000000000000000000 P a45697e94a0f43b7ada7975ad01c6930: Log is configured to *not* fsync() on all Append() calls
I20260808 22:35:32.913918 27082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a45697e94a0f43b7ada7975ad01c6930: No bootstrap required, opened a new log
I20260808 22:35:32.917793 27082 raft_consensus.cc:348] T 00000000000000000000000000000000 P a45697e94a0f43b7ada7975ad01c6930 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a45697e94a0f43b7ada7975ad01c6930" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 39419 } }
I20260808 22:35:32.918021 27082 raft_consensus.cc:374] T 00000000000000000000000000000000 P a45697e94a0f43b7ada7975ad01c6930 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:35:32.918059 27082 raft_consensus.cc:729] T 00000000000000000000000000000000 P a45697e94a0f43b7ada7975ad01c6930 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45697e94a0f43b7ada7975ad01c6930, State: Initialized, Role: FOLLOWER
I20260808 22:35:32.918469 27082 consensus_queue.cc:260] T 00000000000000000000000000000000 P a45697e94a0f43b7ada7975ad01c6930 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a45697e94a0f43b7ada7975ad01c6930" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 39419 } }
I20260808 22:35:32.918617 27082 raft_consensus.cc:388] T 00000000000000000000000000000000 P a45697e94a0f43b7ada7975ad01c6930 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:35:32.918690 27082 raft_consensus.cc:482] T 00000000000000000000000000000000 P a45697e94a0f43b7ada7975ad01c6930 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:35:32.918797 27082 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a45697e94a0f43b7ada7975ad01c6930 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:35:32.919571 27082 raft_consensus.cc:504] T 00000000000000000000000000000000 P a45697e94a0f43b7ada7975ad01c6930 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a45697e94a0f43b7ada7975ad01c6930" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 39419 } }
I20260808 22:35:32.919759 27089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:32.912121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59034 (local address 127.31.87.190:39419)
0808 22:35:32.912679 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:32.912693 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:32.912727 (+ 34us) server_negotiation.cc:407] Connection header received
0808 22:35:32.913366 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:32.913384 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:32.913695 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:32.913950 (+ 255us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:32.914306 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:32.915309 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:32.916486 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:32.916923 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:32.917032 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:32.917951 (+ 919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:32.917957 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:32.917969 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:35:32.918381 (+ 412us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:32.918434 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:35:32.918456 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:32.918531 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:32.918695 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 22:35:32.919060 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":122,"threads_started":1}
I20260808 22:35:32.919888 27082 leader_election.cc:302] T 00000000000000000000000000000000 P a45697e94a0f43b7ada7975ad01c6930 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a45697e94a0f43b7ada7975ad01c6930; no voters:
I20260808 22:35:32.920176 27082 leader_election.cc:288] T 00000000000000000000000000000000 P a45697e94a0f43b7ada7975ad01c6930 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:35:32.920290 27093 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a45697e94a0f43b7ada7975ad01c6930 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:35:32.920454 27093 raft_consensus.cc:686] T 00000000000000000000000000000000 P a45697e94a0f43b7ada7975ad01c6930 [term 1 LEADER]: Becoming Leader. State: Replica: a45697e94a0f43b7ada7975ad01c6930, State: Running, Role: LEADER
I20260808 22:35:32.920733 27093 consensus_queue.cc:237] T 00000000000000000000000000000000 P a45697e94a0f43b7ada7975ad01c6930 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a45697e94a0f43b7ada7975ad01c6930" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 39419 } }
I20260808 22:35:32.921367 27082 sys_catalog.cc:564] T 00000000000000000000000000000000 P a45697e94a0f43b7ada7975ad01c6930 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:35:32.922245 27097 sys_catalog.cc:455] T 00000000000000000000000000000000 P a45697e94a0f43b7ada7975ad01c6930 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a45697e94a0f43b7ada7975ad01c6930" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a45697e94a0f43b7ada7975ad01c6930" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 39419 } } }
I20260808 22:35:32.922382 27097 sys_catalog.cc:458] T 00000000000000000000000000000000 P a45697e94a0f43b7ada7975ad01c6930 [sys.catalog]: This master's current role is: LEADER
I20260808 22:35:32.922658 27098 sys_catalog.cc:455] T 00000000000000000000000000000000 P a45697e94a0f43b7ada7975ad01c6930 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a45697e94a0f43b7ada7975ad01c6930. Latest consensus state: current_term: 1 leader_uuid: "a45697e94a0f43b7ada7975ad01c6930" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a45697e94a0f43b7ada7975ad01c6930" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 39419 } } }
I20260808 22:35:32.922739 27116 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:35:32.922746 27098 sys_catalog.cc:458] T 00000000000000000000000000000000 P a45697e94a0f43b7ada7975ad01c6930 [sys.catalog]: This master's current role is: LEADER
I20260808 22:35:32.925783 27116 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:35:32.931573 27116 catalog_manager.cc:1347] Generated new cluster ID: 11486f5277d14860880675baa607ca28
I20260808 22:35:32.931649 27116 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:35:32.943210 27116 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:35:32.944007 27116 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:35:32.951879 27116 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a45697e94a0f43b7ada7975ad01c6930: Generated new TSK 0
I20260808 22:35:32.952431 27116 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:35:32.957736 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.129:0
--local_ip_for_outbound_sockets=127.31.87.129
--webserver_interface=127.31.87.129
--webserver_port=0
--tserver_master_addrs=127.31.87.190:39419
--builtin_ntp_servers=127.31.87.148:45965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 22:35:33.071831 27131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:35:33.072083 27131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:35:33.072132 27131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:35:33.076035 27131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:35:33.076084 27131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:35:33.076161 27131 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.129
I20260808 22:35:33.080859 27131 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:45965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.87.129
--webserver_port=0
--tserver_master_addrs=127.31.87.190:39419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27131
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.129
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:35:33.081857 27131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:35:33.083134 27131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:35:33.092003 27140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:33.092034 27148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:33.092005 27142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:33.092319 27131 server_base.cc:1034] running on GCE node
I20260808 22:35:33.092667 27131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:35:33.093259 27131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:35:33.094419 27131 hybrid_clock.cc:648] HybridClock initialized: now 1786228533094402 us; error 27 us; skew 500 ppm
I20260808 22:35:33.096120 27131 webserver.cc:460] Webserver started at http://127.31.87.129:33999/ using document root <none> and password file <none>
I20260808 22:35:33.096655 27131 fs_manager.cc:359] Metadata directory not provided
I20260808 22:35:33.096721 27131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:35:33.096903 27131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:35:33.098829 27131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/ts-0/data/instance:
uuid: "a943a6019fa649c18d04a6ec3db45703"
format_stamp: "Formatted at 2026-08-08 22:35:33 on dist-test-slave-266d"
server_key: "25c59d19e2d7c7565e767022a32b4c47"
server_key_iv: "ccd948bee3bc95d6c5e9004750ccf400"
server_key_version: "encryptionkey@0"
I20260808 22:35:33.099354 27131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/ts-0/wal/instance:
uuid: "a943a6019fa649c18d04a6ec3db45703"
format_stamp: "Formatted at 2026-08-08 22:35:33 on dist-test-slave-266d"
server_key: "25c59d19e2d7c7565e767022a32b4c47"
server_key_iv: "ccd948bee3bc95d6c5e9004750ccf400"
server_key_version: "encryptionkey@0"
I20260808 22:35:33.102713 27131 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 22:35:33.104866 27160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:33.105963 27131 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:35:33.106073 27131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/ts-0/wal
uuid: "a943a6019fa649c18d04a6ec3db45703"
format_stamp: "Formatted at 2026-08-08 22:35:33 on dist-test-slave-266d"
server_key: "25c59d19e2d7c7565e767022a32b4c47"
server_key_iv: "ccd948bee3bc95d6c5e9004750ccf400"
server_key_version: "encryptionkey@0"
I20260808 22:35:33.106163 27131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:35:33.123018 27131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:35:33.123611 27131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:35:33.123764 27131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:35:33.124274 27131 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:35:33.125159 27131 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:35:33.125207 27131 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:33.125257 27131 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:35:33.125285 27131 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:33.133271 27131 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.129:35797
I20260808 22:35:33.133299 27330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.129:35797 every 8 connection(s)
I20260808 22:35:33.134091 27131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
I20260808 22:35:33.134799 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 27131
I20260808 22:35:33.134912 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/ts-0/wal/instance
I20260808 22:35:33.135187 32094 external_mini_cluster.cc:1428] Setting key 0fefb733c8fded7c745c5a088901666d
I20260808 22:35:33.137333 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.130:0
--local_ip_for_outbound_sockets=127.31.87.130
--webserver_interface=127.31.87.130
--webserver_port=0
--tserver_master_addrs=127.31.87.190:39419
--builtin_ntp_servers=127.31.87.148:45965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 22:35:33.143216 27089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:33.135790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:48417 (local address 127.31.87.190:39419)
0808 22:35:33.137196 (+ 1406us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:33.137201 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:33.137217 (+ 16us) server_negotiation.cc:407] Connection header received
0808 22:35:33.138441 (+ 1224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:33.138445 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:33.138499 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:33.138593 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:33.139367 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:33.139897 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:33.141395 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:33.141632 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:33.141710 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:33.142802 (+ 1092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:33.142805 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:33.142807 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:35:33.142819 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:33.142838 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:35:33.142846 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:33.142901 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:33.143053 (+ 152us) server_negotiation.cc:299] Negotiation successful
0808 22:35:33.143107 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1321}
I20260808 22:35:33.144243 27335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:33.136230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:39419 (local address 127.31.87.129:48417)
0808 22:35:33.136923 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:33.136962 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 22:35:33.138223 (+ 1261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:35:33.138712 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:35:33.138719 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:35:33.138797 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:35:33.139229 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:33.139240 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:33.140014 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:33.140018 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:33.141259 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:33.141279 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:33.141777 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:33.141781 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:33.141969 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:33.142523 (+ 554us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:35:33.142525 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:35:33.142576 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:35:33.142656 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:35:33.142921 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:35:33.142929 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:35:33.142935 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 22:35:33.143060 (+ 125us) client_negotiation.cc:240] Negotiation successful
0808 22:35:33.143341 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":124,"threads_started":1}
I20260808 22:35:33.145362 27331 heartbeater.cc:347] Connected to a master server at 127.31.87.190:39419
I20260808 22:35:33.145591 27331 heartbeater.cc:464] Registering TS with master...
I20260808 22:35:33.146047 27331 heartbeater.cc:511] Master 127.31.87.190:39419 requested a full tablet report, sending...
I20260808 22:35:33.147380 27036 ts_manager.cc:194] Registered new tserver with Master: a943a6019fa649c18d04a6ec3db45703 (127.31.87.129:35797)
I20260808 22:35:33.148478 27036 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.87.129:48417
W20260808 22:35:33.244205 27337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:35:33.244434 27337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:35:33.244483 27337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:35:33.248337 27337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:35:33.248380 27337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:35:33.248449 27337 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.130
I20260808 22:35:33.252926 27337 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:45965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.87.130
--webserver_port=0
--tserver_master_addrs=127.31.87.190:39419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27337
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.130
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:35:33.253844 27337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:35:33.255072 27337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:35:33.263881 27349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:33.263892 27344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:33.263991 27337 server_base.cc:1034] running on GCE node
W20260808 22:35:33.263892 27345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:33.264503 27337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:35:33.265118 27337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:35:33.266279 27337 hybrid_clock.cc:648] HybridClock initialized: now 1786228533266263 us; error 27 us; skew 500 ppm
I20260808 22:35:33.267964 27337 webserver.cc:460] Webserver started at http://127.31.87.130:36051/ using document root <none> and password file <none>
I20260808 22:35:33.268483 27337 fs_manager.cc:359] Metadata directory not provided
I20260808 22:35:33.268543 27337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:35:33.268734 27337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:35:33.270677 27337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/ts-1/data/instance:
uuid: "baf5889102e043e9b0d12bc8bfb8ecb2"
format_stamp: "Formatted at 2026-08-08 22:35:33 on dist-test-slave-266d"
server_key: "4711a166caacb78fc652423a33b467cd"
server_key_iv: "aae6a7d9a34701e39fb75692fdca9751"
server_key_version: "encryptionkey@0"
I20260808 22:35:33.271181 27337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/ts-1/wal/instance:
uuid: "baf5889102e043e9b0d12bc8bfb8ecb2"
format_stamp: "Formatted at 2026-08-08 22:35:33 on dist-test-slave-266d"
server_key: "4711a166caacb78fc652423a33b467cd"
server_key_iv: "aae6a7d9a34701e39fb75692fdca9751"
server_key_version: "encryptionkey@0"
I20260808 22:35:33.274488 27337 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20260808 22:35:33.276659 27359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:33.277824 27337 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260808 22:35:33.277921 27337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/ts-1/wal
uuid: "baf5889102e043e9b0d12bc8bfb8ecb2"
format_stamp: "Formatted at 2026-08-08 22:35:33 on dist-test-slave-266d"
server_key: "4711a166caacb78fc652423a33b467cd"
server_key_iv: "aae6a7d9a34701e39fb75692fdca9751"
server_key_version: "encryptionkey@0"
I20260808 22:35:33.278007 27337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:35:33.294996 27337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:35:33.295557 27337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:35:33.295722 27337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:35:33.296228 27337 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:35:33.297082 27337 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:35:33.297164 27337 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:33.297216 27337 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:35:33.297242 27337 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:33.305186 27337 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.130:45137
I20260808 22:35:33.305207 27527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.130:45137 every 8 connection(s)
I20260808 22:35:33.305971 27337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
I20260808 22:35:33.313990 27089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:33.307566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:54343 (local address 127.31.87.190:39419)
0808 22:35:33.307697 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:33.307701 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:33.308645 (+ 944us) server_negotiation.cc:407] Connection header received
0808 22:35:33.310027 (+ 1382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:33.310030 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:33.310076 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:33.310148 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:33.310910 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:33.311409 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:33.312424 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:33.312617 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:33.312678 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:33.313597 (+ 919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:33.313601 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:33.313603 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:35:33.313615 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:33.313635 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:35:33.313643 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:33.313698 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:33.313796 (+ 98us) server_negotiation.cc:299] Negotiation successful
0808 22:35:33.313845 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 22:35:33.314118 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 27337
I20260808 22:35:33.314230 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/ts-1/wal/instance
I20260808 22:35:33.314502 32094 external_mini_cluster.cc:1428] Setting key 6d3b8b4ce0869da5ec786810199e4de7
I20260808 22:35:33.314751 27531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:33.307894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:39419 (local address 127.31.87.130:54343)
0808 22:35:33.308491 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:33.308529 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 22:35:33.309849 (+ 1320us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:35:33.310281 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:35:33.310288 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:35:33.310366 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:35:33.310807 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:33.310819 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:33.311493 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:33.311496 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:33.312334 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:33.312342 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:33.312713 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:33.312715 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:33.312842 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:33.313363 (+ 521us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:35:33.313364 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:35:33.313401 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:35:33.313460 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:35:33.313703 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:35:33.313707 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:35:33.313711 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:35:33.313791 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 22:35:33.313985 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":109,"threads_started":1}
I20260808 22:35:33.315836 27528 heartbeater.cc:347] Connected to a master server at 127.31.87.190:39419
I20260808 22:35:33.316048 27528 heartbeater.cc:464] Registering TS with master...
I20260808 22:35:33.316448 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.131:0
--local_ip_for_outbound_sockets=127.31.87.131
--webserver_interface=127.31.87.131
--webserver_port=0
--tserver_master_addrs=127.31.87.190:39419
--builtin_ntp_servers=127.31.87.148:45965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 22:35:33.316606 27528 heartbeater.cc:511] Master 127.31.87.190:39419 requested a full tablet report, sending...
I20260808 22:35:33.317621 27036 ts_manager.cc:194] Registered new tserver with Master: baf5889102e043e9b0d12bc8bfb8ecb2 (127.31.87.130:45137)
I20260808 22:35:33.318292 27036 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.87.130:54343
W20260808 22:35:33.424876 27532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:35:33.425148 27532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:35:33.425210 27532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:35:33.429210 27532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:35:33.429262 27532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:35:33.429345 27532 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.131
I20260808 22:35:33.434257 27532 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:45965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.87.131
--webserver_port=0
--tserver_master_addrs=127.31.87.190:39419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27532
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.131
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:35:33.435204 27532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:35:33.436546 27532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:35:33.445643 27544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:33.445741 27540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:33.445643 27537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:33.445838 27532 server_base.cc:1034] running on GCE node
I20260808 22:35:33.446286 27532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:35:33.446817 27532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:35:33.447969 27532 hybrid_clock.cc:648] HybridClock initialized: now 1786228533447953 us; error 32 us; skew 500 ppm
I20260808 22:35:33.449678 27532 webserver.cc:460] Webserver started at http://127.31.87.131:36039/ using document root <none> and password file <none>
I20260808 22:35:33.450191 27532 fs_manager.cc:359] Metadata directory not provided
I20260808 22:35:33.450253 27532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:35:33.450449 27532 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:35:33.452395 27532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/ts-2/data/instance:
uuid: "649d19608c004f209abcc801b62026a1"
format_stamp: "Formatted at 2026-08-08 22:35:33 on dist-test-slave-266d"
server_key: "3272a4159964b14aefaceba75062af1b"
server_key_iv: "0a671dd6342bbe3896c70bceedf75110"
server_key_version: "encryptionkey@0"
I20260808 22:35:33.452896 27532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/ts-2/wal/instance:
uuid: "649d19608c004f209abcc801b62026a1"
format_stamp: "Formatted at 2026-08-08 22:35:33 on dist-test-slave-266d"
server_key: "3272a4159964b14aefaceba75062af1b"
server_key_iv: "0a671dd6342bbe3896c70bceedf75110"
server_key_version: "encryptionkey@0"
I20260808 22:35:33.456184 27532 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 22:35:33.458405 27554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:33.459488 27532 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:35:33.459585 27532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/ts-2/wal
uuid: "649d19608c004f209abcc801b62026a1"
format_stamp: "Formatted at 2026-08-08 22:35:33 on dist-test-slave-266d"
server_key: "3272a4159964b14aefaceba75062af1b"
server_key_iv: "0a671dd6342bbe3896c70bceedf75110"
server_key_version: "encryptionkey@0"
I20260808 22:35:33.459668 27532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:35:33.486137 27532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:35:33.486742 27532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:35:33.486907 27532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:35:33.487428 27532 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:35:33.488297 27532 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:35:33.488343 27532 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:33.488391 27532 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:35:33.488420 27532 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:33.496608 27532 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.131:39809
I20260808 22:35:33.496639 27731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.131:39809 every 8 connection(s)
I20260808 22:35:33.497443 27532 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
I20260808 22:35:33.502822 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 27532
I20260808 22:35:33.502933 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/ts-2/wal/instance
I20260808 22:35:33.503201 32094 external_mini_cluster.cc:1428] Setting key 18588e3fb34e9b60c586c18d7a488531
I20260808 22:35:33.506080 27089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:33.499077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:48749 (local address 127.31.87.190:39419)
0808 22:35:33.499193 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:33.499197 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:33.500141 (+ 944us) server_negotiation.cc:407] Connection header received
0808 22:35:33.501512 (+ 1371us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:33.501515 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:33.501576 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:33.501648 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:33.502482 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:33.503197 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:33.504330 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:33.504543 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:33.504641 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:33.505641 (+ 1000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:33.505645 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:33.505648 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:35:33.505660 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:33.505678 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:35:33.505685 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:33.505744 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:33.505886 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 22:35:33.505946 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 22:35:33.506886 27735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:33.499406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:39419 (local address 127.31.87.131:48749)
0808 22:35:33.500007 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:33.500044 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 22:35:33.501317 (+ 1273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:35:33.501822 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:35:33.501829 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:35:33.501908 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:35:33.502348 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:33.502360 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:33.503337 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:33.503341 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:33.504202 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:33.504212 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:33.504676 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:33.504680 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:33.504832 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:33.505393 (+ 561us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:35:33.505395 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:35:33.505431 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:35:33.505490 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:35:33.505754 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:35:33.505763 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:35:33.505769 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 22:35:33.505874 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 22:35:33.506139 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":124,"threads_started":1}
I20260808 22:35:33.507797 27733 heartbeater.cc:347] Connected to a master server at 127.31.87.190:39419
I20260808 22:35:33.507984 27733 heartbeater.cc:464] Registering TS with master...
I20260808 22:35:33.508438 27733 heartbeater.cc:511] Master 127.31.87.190:39419 requested a full tablet report, sending...
I20260808 22:35:33.509337 27036 ts_manager.cc:194] Registered new tserver with Master: 649d19608c004f209abcc801b62026a1 (127.31.87.131:39809)
I20260808 22:35:33.509838 27036 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.87.131:48749
I20260808 22:35:33.516470 32094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:35:33.521478 27089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:33.518320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59042 (local address 127.31.87.190:39419)
0808 22:35:33.518432 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:33.518435 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:33.518603 (+ 168us) server_negotiation.cc:407] Connection header received
0808 22:35:33.518652 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:33.518654 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:33.518698 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:33.518770 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:33.518999 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:33.519484 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:33.520214 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:33.520412 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:33.520483 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:33.521140 (+ 657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:33.521144 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:33.521147 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:35:33.521162 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:33.521182 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:35:33.521191 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:33.521256 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:33.521319 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 22:35:33.521377 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 22:35:33.523969 27036 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59042:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:35:33.526971 27036 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:35:33.540267 27751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:33.534823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.130:45137 (local address 127.0.0.1:45724)
0808 22:35:33.535142 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:33.535185 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 22:35:33.535309 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:35:33.535901 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:35:33.535908 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:35:33.535935 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:35:33.536075 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:33.536083 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:33.537610 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:33.537613 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:33.538595 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:33.538606 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:33.539644 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:33.539648 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:33.539818 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:33.539859 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 22:35:33.539977 (+ 118us) client_negotiation.cc:240] Negotiation successful
0808 22:35:33.540107 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":24,"thread_start_us":81,"threads_started":1}
I20260808 22:35:33.540307 27753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:33.534976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45724 (local address 127.31.87.130:45137)
0808 22:35:33.535333 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:33.535347 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:33.535377 (+ 30us) server_negotiation.cc:407] Connection header received
0808 22:35:33.535463 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:33.535469 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:33.535696 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:33.536256 (+ 560us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:35:33.536435 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:33.537513 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:33.538739 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:33.539529 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:33.539628 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:33.539817 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:33.540017 (+ 200us) server_negotiation.cc:299] Negotiation successful
0808 22:35:33.540119 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":142,"threads_started":1}
I20260808 22:35:33.540786 27758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:33.535496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:35797 (local address 127.0.0.1:44336)
0808 22:35:33.535734 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:33.535746 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:35:33.535826 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:35:33.536798 (+ 972us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:35:33.536801 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:35:33.536823 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:35:33.536928 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:33.536934 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:33.538342 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:33.538346 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:33.539381 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:33.539392 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:33.540441 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:33.540446 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:33.540538 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:33.540576 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 22:35:33.540632 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 22:35:33.540688 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":1,"thread_start_us":98,"threads_started":1}
I20260808 22:35:33.541533 27759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:33.535937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44336 (local address 127.31.87.129:35797)
0808 22:35:33.536304 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:33.536328 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:33.536352 (+ 24us) server_negotiation.cc:407] Connection header received
0808 22:35:33.536428 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:33.536433 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:33.536617 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:33.536806 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:35:33.537060 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:33.538212 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:33.539525 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:33.540292 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:33.540976 (+ 684us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:33.541203 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:33.541261 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 22:35:33.541356 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":150,"threads_started":1}
I20260808 22:35:33.541651 27756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:33.535503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.131:39809 (local address 127.0.0.1:32794)
0808 22:35:33.536533 (+ 1030us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:33.536545 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:35:33.536705 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:35:33.537181 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:35:33.537185 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:35:33.537204 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:35:33.537291 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:33.537297 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:33.538536 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:33.538539 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:33.539714 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:33.539725 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:33.541319 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:33.541322 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:33.541382 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:33.541396 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 22:35:33.541452 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 22:35:33.541515 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":972,"mutex_wait_us":54,"thread_start_us":95,"threads_started":1}
I20260808 22:35:33.541731 27761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:33.535989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32794 (local address 127.31.87.131:39809)
0808 22:35:33.536392 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:33.536402 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:33.536637 (+ 235us) server_negotiation.cc:407] Connection header received
0808 22:35:33.536860 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:33.536865 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:33.537032 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:33.537192 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:35:33.537396 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:33.538440 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:33.539857 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:33.540966 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:33.541045 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:33.541232 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:33.541505 (+ 273us) server_negotiation.cc:299] Negotiation successful
0808 22:35:33.541592 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":113,"threads_started":1}
I20260808 22:35:33.543231 27420 tablet_service.cc:1467] Processing CreateTablet for tablet 7b62be4a374a45ee816888d0800b4890 (DEFAULT_TABLE table=test-table [id=5cf1c348c05d445db48b81280b43ccb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:35:33.544152 27420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b62be4a374a45ee816888d0800b4890. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:35:33.544018 27626 tablet_service.cc:1467] Processing CreateTablet for tablet 7b62be4a374a45ee816888d0800b4890 (DEFAULT_TABLE table=test-table [id=5cf1c348c05d445db48b81280b43ccb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:35:33.544802 27626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b62be4a374a45ee816888d0800b4890. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:35:33.547719 27226 tablet_service.cc:1467] Processing CreateTablet for tablet 7b62be4a374a45ee816888d0800b4890 (DEFAULT_TABLE table=test-table [id=5cf1c348c05d445db48b81280b43ccb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:35:33.548774 27226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b62be4a374a45ee816888d0800b4890. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:35:33.551395 27763 tablet_bootstrap.cc:492] T 7b62be4a374a45ee816888d0800b4890 P 649d19608c004f209abcc801b62026a1: Bootstrap starting.
I20260808 22:35:33.553442 27764 tablet_bootstrap.cc:492] T 7b62be4a374a45ee816888d0800b4890 P baf5889102e043e9b0d12bc8bfb8ecb2: Bootstrap starting.
I20260808 22:35:33.553514 27763 tablet_bootstrap.cc:654] T 7b62be4a374a45ee816888d0800b4890 P 649d19608c004f209abcc801b62026a1: Neither blocks nor log segments found. Creating new log.
I20260808 22:35:33.554356 27763 log.cc:824] T 7b62be4a374a45ee816888d0800b4890 P 649d19608c004f209abcc801b62026a1: Log is configured to *not* fsync() on all Append() calls
I20260808 22:35:33.555073 27768 tablet_bootstrap.cc:492] T 7b62be4a374a45ee816888d0800b4890 P a943a6019fa649c18d04a6ec3db45703: Bootstrap starting.
I20260808 22:35:33.555555 27764 tablet_bootstrap.cc:654] T 7b62be4a374a45ee816888d0800b4890 P baf5889102e043e9b0d12bc8bfb8ecb2: Neither blocks nor log segments found. Creating new log.
I20260808 22:35:33.556413 27764 log.cc:824] T 7b62be4a374a45ee816888d0800b4890 P baf5889102e043e9b0d12bc8bfb8ecb2: Log is configured to *not* fsync() on all Append() calls
I20260808 22:35:33.556519 27763 tablet_bootstrap.cc:492] T 7b62be4a374a45ee816888d0800b4890 P 649d19608c004f209abcc801b62026a1: No bootstrap required, opened a new log
I20260808 22:35:33.556702 27763 ts_tablet_manager.cc:1397] T 7b62be4a374a45ee816888d0800b4890 P 649d19608c004f209abcc801b62026a1: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 22:35:33.558418 27764 tablet_bootstrap.cc:492] T 7b62be4a374a45ee816888d0800b4890 P baf5889102e043e9b0d12bc8bfb8ecb2: No bootstrap required, opened a new log
I20260808 22:35:33.558605 27764 ts_tablet_manager.cc:1397] T 7b62be4a374a45ee816888d0800b4890 P baf5889102e043e9b0d12bc8bfb8ecb2: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 22:35:33.560918 27763 raft_consensus.cc:348] T 7b62be4a374a45ee816888d0800b4890 P 649d19608c004f209abcc801b62026a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baf5889102e043e9b0d12bc8bfb8ecb2" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 45137 } } peers { permanent_uuid: "649d19608c004f209abcc801b62026a1" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 39809 } } peers { permanent_uuid: "a943a6019fa649c18d04a6ec3db45703" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 35797 } }
I20260808 22:35:33.561213 27763 raft_consensus.cc:374] T 7b62be4a374a45ee816888d0800b4890 P 649d19608c004f209abcc801b62026a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:35:33.561269 27763 raft_consensus.cc:729] T 7b62be4a374a45ee816888d0800b4890 P 649d19608c004f209abcc801b62026a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 649d19608c004f209abcc801b62026a1, State: Initialized, Role: FOLLOWER
I20260808 22:35:33.561728 27763 consensus_queue.cc:260] T 7b62be4a374a45ee816888d0800b4890 P 649d19608c004f209abcc801b62026a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baf5889102e043e9b0d12bc8bfb8ecb2" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 45137 } } peers { permanent_uuid: "649d19608c004f209abcc801b62026a1" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 39809 } } peers { permanent_uuid: "a943a6019fa649c18d04a6ec3db45703" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 35797 } }
I20260808 22:35:33.562428 27733 heartbeater.cc:503] Master 127.31.87.190:39419 was elected leader, sending a full tablet report...
I20260808 22:35:33.562737 27763 ts_tablet_manager.cc:1428] T 7b62be4a374a45ee816888d0800b4890 P 649d19608c004f209abcc801b62026a1: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260808 22:35:33.563297 27764 raft_consensus.cc:348] T 7b62be4a374a45ee816888d0800b4890 P baf5889102e043e9b0d12bc8bfb8ecb2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baf5889102e043e9b0d12bc8bfb8ecb2" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 45137 } } peers { permanent_uuid: "649d19608c004f209abcc801b62026a1" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 39809 } } peers { permanent_uuid: "a943a6019fa649c18d04a6ec3db45703" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 35797 } }
I20260808 22:35:33.563544 27764 raft_consensus.cc:374] T 7b62be4a374a45ee816888d0800b4890 P baf5889102e043e9b0d12bc8bfb8ecb2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:35:33.563596 27764 raft_consensus.cc:729] T 7b62be4a374a45ee816888d0800b4890 P baf5889102e043e9b0d12bc8bfb8ecb2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: baf5889102e043e9b0d12bc8bfb8ecb2, State: Initialized, Role: FOLLOWER
I20260808 22:35:33.563683 27768 tablet_bootstrap.cc:654] T 7b62be4a374a45ee816888d0800b4890 P a943a6019fa649c18d04a6ec3db45703: Neither blocks nor log segments found. Creating new log.
I20260808 22:35:33.564041 27764 consensus_queue.cc:260] T 7b62be4a374a45ee816888d0800b4890 P baf5889102e043e9b0d12bc8bfb8ecb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baf5889102e043e9b0d12bc8bfb8ecb2" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 45137 } } peers { permanent_uuid: "649d19608c004f209abcc801b62026a1" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 39809 } } peers { permanent_uuid: "a943a6019fa649c18d04a6ec3db45703" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 35797 } }
I20260808 22:35:33.564602 27768 log.cc:824] T 7b62be4a374a45ee816888d0800b4890 P a943a6019fa649c18d04a6ec3db45703: Log is configured to *not* fsync() on all Append() calls
I20260808 22:35:33.565007 27764 ts_tablet_manager.cc:1428] T 7b62be4a374a45ee816888d0800b4890 P baf5889102e043e9b0d12bc8bfb8ecb2: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 22:35:33.565224 27528 heartbeater.cc:503] Master 127.31.87.190:39419 was elected leader, sending a full tablet report...
I20260808 22:35:33.566707 27768 tablet_bootstrap.cc:492] T 7b62be4a374a45ee816888d0800b4890 P a943a6019fa649c18d04a6ec3db45703: No bootstrap required, opened a new log
I20260808 22:35:33.566962 27768 ts_tablet_manager.cc:1397] T 7b62be4a374a45ee816888d0800b4890 P a943a6019fa649c18d04a6ec3db45703: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.000s
I20260808 22:35:33.570314 27768 raft_consensus.cc:348] T 7b62be4a374a45ee816888d0800b4890 P a943a6019fa649c18d04a6ec3db45703 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baf5889102e043e9b0d12bc8bfb8ecb2" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 45137 } } peers { permanent_uuid: "649d19608c004f209abcc801b62026a1" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 39809 } } peers { permanent_uuid: "a943a6019fa649c18d04a6ec3db45703" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 35797 } }
I20260808 22:35:33.570508 27768 raft_consensus.cc:374] T 7b62be4a374a45ee816888d0800b4890 P a943a6019fa649c18d04a6ec3db45703 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:35:33.570554 27768 raft_consensus.cc:729] T 7b62be4a374a45ee816888d0800b4890 P a943a6019fa649c18d04a6ec3db45703 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a943a6019fa649c18d04a6ec3db45703, State: Initialized, Role: FOLLOWER
I20260808 22:35:33.570873 27768 consensus_queue.cc:260] T 7b62be4a374a45ee816888d0800b4890 P a943a6019fa649c18d04a6ec3db45703 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baf5889102e043e9b0d12bc8bfb8ecb2" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 45137 } } peers { permanent_uuid: "649d19608c004f209abcc801b62026a1" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 39809 } } peers { permanent_uuid: "a943a6019fa649c18d04a6ec3db45703" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 35797 } }
I20260808 22:35:33.571414 27331 heartbeater.cc:503] Master 127.31.87.190:39419 was elected leader, sending a full tablet report...
I20260808 22:35:33.571605 27768 ts_tablet_manager.cc:1428] T 7b62be4a374a45ee816888d0800b4890 P a943a6019fa649c18d04a6ec3db45703: Time spent starting tablet: real 0.005s user 0.000s sys 0.004s
I20260808 22:35:33.617812 27778 raft_consensus.cc:482] T 7b62be4a374a45ee816888d0800b4890 P a943a6019fa649c18d04a6ec3db45703 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:35:33.618021 27778 raft_consensus.cc:504] T 7b62be4a374a45ee816888d0800b4890 P a943a6019fa649c18d04a6ec3db45703 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baf5889102e043e9b0d12bc8bfb8ecb2" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 45137 } } peers { permanent_uuid: "649d19608c004f209abcc801b62026a1" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 39809 } } peers { permanent_uuid: "a943a6019fa649c18d04a6ec3db45703" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 35797 } }
I20260808 22:35:33.619141 27778 leader_election.cc:288] T 7b62be4a374a45ee816888d0800b4890 P a943a6019fa649c18d04a6ec3db45703 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers baf5889102e043e9b0d12bc8bfb8ecb2 (127.31.87.130:45137), 649d19608c004f209abcc801b62026a1 (127.31.87.131:39809)
I20260808 22:35:33.622803 27753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:33.619458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:43735 (local address 127.31.87.130:45137)
0808 22:35:33.619606 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:33.619610 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:33.619649 (+ 39us) server_negotiation.cc:407] Connection header received
0808 22:35:33.619706 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:33.619709 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:33.619761 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:33.619831 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:35:33.620149 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:33.620637 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:33.621684 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:33.622260 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:33.622364 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:33.622413 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:33.622602 (+ 189us) server_negotiation.cc:299] Negotiation successful
0808 22:35:33.622683 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 22:35:33.622810 27335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:33.619335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.130:45137 (local address 127.31.87.129:43735)
0808 22:35:33.619479 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:33.619496 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 22:35:33.619593 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:35:33.619914 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:35:33.619917 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:35:33.619937 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:35:33.620030 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:33.620035 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:33.620738 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:33.620742 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:33.621559 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:33.621570 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:33.622385 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:33.622389 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:33.622487 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:33.622514 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 22:35:33.622590 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 22:35:33.622678 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 22:35:33.623577 27761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:33.619726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:42499 (local address 127.31.87.131:39809)
0808 22:35:33.619854 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:33.619858 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:33.620015 (+ 157us) server_negotiation.cc:407] Connection header received
0808 22:35:33.620127 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:33.620130 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:33.620174 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:33.620234 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:35:33.620508 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:33.620990 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:33.622280 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:33.622879 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:33.622993 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:33.623040 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:33.623407 (+ 367us) server_negotiation.cc:299] Negotiation successful
0808 22:35:33.623487 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 22:35:33.623577 27780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:33.619627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.131:39809 (local address 127.31.87.129:42499)
0808 22:35:33.619939 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:33.619950 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:35:33.620037 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:35:33.620254 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:35:33.620257 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:35:33.620282 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:35:33.620410 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:33.620415 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:33.621111 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:33.621114 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:33.622161 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:33.622172 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:33.623134 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:33.623137 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:33.623213 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:33.623249 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 22:35:33.623385 (+ 136us) client_negotiation.cc:240] Negotiation successful
0808 22:35:33.623452 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":105,"threads_started":1}
I20260808 22:35:33.623569 27452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b62be4a374a45ee816888d0800b4890" candidate_uuid: "a943a6019fa649c18d04a6ec3db45703" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baf5889102e043e9b0d12bc8bfb8ecb2" is_pre_election: true
I20260808 22:35:33.623891 27452 raft_consensus.cc:2454] T 7b62be4a374a45ee816888d0800b4890 P baf5889102e043e9b0d12bc8bfb8ecb2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a943a6019fa649c18d04a6ec3db45703 in term 0.
I20260808 22:35:33.624128 27655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b62be4a374a45ee816888d0800b4890" candidate_uuid: "a943a6019fa649c18d04a6ec3db45703" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "649d19608c004f209abcc801b62026a1" is_pre_election: true
I20260808 22:35:33.624428 27655 raft_consensus.cc:2454] T 7b62be4a374a45ee816888d0800b4890 P 649d19608c004f209abcc801b62026a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a943a6019fa649c18d04a6ec3db45703 in term 0.
I20260808 22:35:33.624406 27162 leader_election.cc:302] T 7b62be4a374a45ee816888d0800b4890 P a943a6019fa649c18d04a6ec3db45703 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a943a6019fa649c18d04a6ec3db45703, baf5889102e043e9b0d12bc8bfb8ecb2; no voters:
I20260808 22:35:33.624696 27778 raft_consensus.cc:2781] T 7b62be4a374a45ee816888d0800b4890 P a943a6019fa649c18d04a6ec3db45703 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:35:33.624812 27778 raft_consensus.cc:482] T 7b62be4a374a45ee816888d0800b4890 P a943a6019fa649c18d04a6ec3db45703 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:35:33.624869 27778 raft_consensus.cc:3037] T 7b62be4a374a45ee816888d0800b4890 P a943a6019fa649c18d04a6ec3db45703 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:35:33.625890 27778 raft_consensus.cc:504] T 7b62be4a374a45ee816888d0800b4890 P a943a6019fa649c18d04a6ec3db45703 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baf5889102e043e9b0d12bc8bfb8ecb2" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 45137 } } peers { permanent_uuid: "649d19608c004f209abcc801b62026a1" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 39809 } } peers { permanent_uuid: "a943a6019fa649c18d04a6ec3db45703" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 35797 } }
I20260808 22:35:33.626216 27778 leader_election.cc:288] T 7b62be4a374a45ee816888d0800b4890 P a943a6019fa649c18d04a6ec3db45703 [CANDIDATE]: Term 1 election: Requested vote from peers baf5889102e043e9b0d12bc8bfb8ecb2 (127.31.87.130:45137), 649d19608c004f209abcc801b62026a1 (127.31.87.131:39809)
I20260808 22:35:33.626585 27655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b62be4a374a45ee816888d0800b4890" candidate_uuid: "a943a6019fa649c18d04a6ec3db45703" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "649d19608c004f209abcc801b62026a1"
I20260808 22:35:33.626585 27452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b62be4a374a45ee816888d0800b4890" candidate_uuid: "a943a6019fa649c18d04a6ec3db45703" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baf5889102e043e9b0d12bc8bfb8ecb2"
I20260808 22:35:33.626699 27655 raft_consensus.cc:3037] T 7b62be4a374a45ee816888d0800b4890 P 649d19608c004f209abcc801b62026a1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:35:33.626727 27452 raft_consensus.cc:3037] T 7b62be4a374a45ee816888d0800b4890 P baf5889102e043e9b0d12bc8bfb8ecb2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:35:33.627665 27452 raft_consensus.cc:2454] T 7b62be4a374a45ee816888d0800b4890 P baf5889102e043e9b0d12bc8bfb8ecb2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a943a6019fa649c18d04a6ec3db45703 in term 1.
I20260808 22:35:33.627687 27655 raft_consensus.cc:2454] T 7b62be4a374a45ee816888d0800b4890 P 649d19608c004f209abcc801b62026a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a943a6019fa649c18d04a6ec3db45703 in term 1.
I20260808 22:35:33.627943 27162 leader_election.cc:302] T 7b62be4a374a45ee816888d0800b4890 P a943a6019fa649c18d04a6ec3db45703 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a943a6019fa649c18d04a6ec3db45703, baf5889102e043e9b0d12bc8bfb8ecb2; no voters:
I20260808 22:35:33.628114 27778 raft_consensus.cc:2781] T 7b62be4a374a45ee816888d0800b4890 P a943a6019fa649c18d04a6ec3db45703 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:35:33.628341 27778 raft_consensus.cc:686] T 7b62be4a374a45ee816888d0800b4890 P a943a6019fa649c18d04a6ec3db45703 [term 1 LEADER]: Becoming Leader. State: Replica: a943a6019fa649c18d04a6ec3db45703, State: Running, Role: LEADER
I20260808 22:35:33.628677 27778 consensus_queue.cc:237] T 7b62be4a374a45ee816888d0800b4890 P a943a6019fa649c18d04a6ec3db45703 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baf5889102e043e9b0d12bc8bfb8ecb2" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 45137 } } peers { permanent_uuid: "649d19608c004f209abcc801b62026a1" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 39809 } } peers { permanent_uuid: "a943a6019fa649c18d04a6ec3db45703" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 35797 } }
I20260808 22:35:33.631824 27034 catalog_manager.cc:5495] T 7b62be4a374a45ee816888d0800b4890 P a943a6019fa649c18d04a6ec3db45703 reported cstate change: term changed from 0 to 1, leader changed from <none> to a943a6019fa649c18d04a6ec3db45703 (127.31.87.129). New cstate: current_term: 1 leader_uuid: "a943a6019fa649c18d04a6ec3db45703" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baf5889102e043e9b0d12bc8bfb8ecb2" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 45137 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "649d19608c004f209abcc801b62026a1" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 39809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a943a6019fa649c18d04a6ec3db45703" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 35797 } health_report { overall_health: HEALTHY } } }
W20260808 22:35:33.635172 27333 tablet.cc:2369] T 7b62be4a374a45ee816888d0800b4890 P a943a6019fa649c18d04a6ec3db45703: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:35:33.660493 27759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:33.657066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44348 (local address 127.31.87.129:35797)
0808 22:35:33.657196 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:33.657201 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:33.657243 (+ 42us) server_negotiation.cc:407] Connection header received
0808 22:35:33.657418 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:33.657420 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:33.657473 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:33.657561 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:35:33.657828 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:33.658314 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:33.659155 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:33.659362 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:33.659447 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:33.659745 (+ 298us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:35:33.660135 (+ 390us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:35:33.660240 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:33.660320 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 22:35:33.660370 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 22:35:33.666009 27655 raft_consensus.cc:1261] T 7b62be4a374a45ee816888d0800b4890 P 649d19608c004f209abcc801b62026a1 [term 1 FOLLOWER]: Refusing update from remote peer a943a6019fa649c18d04a6ec3db45703: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:35:33.666010 27452 raft_consensus.cc:1261] T 7b62be4a374a45ee816888d0800b4890 P baf5889102e043e9b0d12bc8bfb8ecb2 [term 1 FOLLOWER]: Refusing update from remote peer a943a6019fa649c18d04a6ec3db45703: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:35:33.666620 27778 consensus_queue.cc:1035] T 7b62be4a374a45ee816888d0800b4890 P a943a6019fa649c18d04a6ec3db45703 [LEADER]: Connected to new peer: Peer: permanent_uuid: "649d19608c004f209abcc801b62026a1" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:35:33.666850 27783 consensus_queue.cc:1035] T 7b62be4a374a45ee816888d0800b4890 P a943a6019fa649c18d04a6ec3db45703 [LEADER]: Connected to new peer: Peer: permanent_uuid: "baf5889102e043e9b0d12bc8bfb8ecb2" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 45137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:35:33.673161 27790 mvcc.cc:205] Tried to move back new op lower bound from 7316392073889263616 to 7316392073746120704. Current Snapshot: MvccSnapshot[applied={T|T < 7316392073889263616}]
I20260808 22:35:33.676777 27792 mvcc.cc:205] Tried to move back new op lower bound from 7316392073889263616 to 7316392073746120704. Current Snapshot: MvccSnapshot[applied={T|T < 7316392073889263616}]
I20260808 22:35:33.681452 27036 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59042:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:35:33.681624 27036 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59042:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:35:33.683843 27036 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a45697e94a0f43b7ada7975ad01c6930: Sending DeleteTablet for 3 replicas of tablet 7b62be4a374a45ee816888d0800b4890
I20260808 22:35:33.684597 27226 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b62be4a374a45ee816888d0800b4890 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:35:33 UTC) from {username='slave'} at 127.0.0.1:44336
I20260808 22:35:33.684624 27626 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b62be4a374a45ee816888d0800b4890 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:35:33 UTC) from {username='slave'} at 127.0.0.1:32794
I20260808 22:35:33.684743 27420 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b62be4a374a45ee816888d0800b4890 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:35:33 UTC) from {username='slave'} at 127.0.0.1:45724
I20260808 22:35:33.685074 27807 tablet_replica.cc:330] T 7b62be4a374a45ee816888d0800b4890 P 649d19608c004f209abcc801b62026a1: stopping tablet replica
I20260808 22:35:33.685129 27810 tablet_replica.cc:330] T 7b62be4a374a45ee816888d0800b4890 P baf5889102e043e9b0d12bc8bfb8ecb2: stopping tablet replica
I20260808 22:35:33.685380 27810 raft_consensus.cc:2229] T 7b62be4a374a45ee816888d0800b4890 P baf5889102e043e9b0d12bc8bfb8ecb2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:35:33.685380 27807 raft_consensus.cc:2229] T 7b62be4a374a45ee816888d0800b4890 P 649d19608c004f209abcc801b62026a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:35:33.685654 27807 raft_consensus.cc:2258] T 7b62be4a374a45ee816888d0800b4890 P 649d19608c004f209abcc801b62026a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:35:33.685654 27810 raft_consensus.cc:2258] T 7b62be4a374a45ee816888d0800b4890 P baf5889102e043e9b0d12bc8bfb8ecb2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:35:33.685870 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 27131
I20260808 22:35:33.691983 27810 ts_tablet_manager.cc:1905] T 7b62be4a374a45ee816888d0800b4890 P baf5889102e043e9b0d12bc8bfb8ecb2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:35:33.692045 27807 ts_tablet_manager.cc:1905] T 7b62be4a374a45ee816888d0800b4890 P 649d19608c004f209abcc801b62026a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:35:33.694638 27807 ts_tablet_manager.cc:1918] T 7b62be4a374a45ee816888d0800b4890 P 649d19608c004f209abcc801b62026a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:35:33.694639 27810 ts_tablet_manager.cc:1918] T 7b62be4a374a45ee816888d0800b4890 P baf5889102e043e9b0d12bc8bfb8ecb2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:35:33.694737 27810 log.cc:1196] T 7b62be4a374a45ee816888d0800b4890 P baf5889102e043e9b0d12bc8bfb8ecb2: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/ts-1/wal/wals/7b62be4a374a45ee816888d0800b4890
I20260808 22:35:33.694737 27807 log.cc:1196] T 7b62be4a374a45ee816888d0800b4890 P 649d19608c004f209abcc801b62026a1: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786228291013016-32094-0/minicluster-data/ts-2/wal/wals/7b62be4a374a45ee816888d0800b4890
I20260808 22:35:33.695035 27810 ts_tablet_manager.cc:1939] T 7b62be4a374a45ee816888d0800b4890 P baf5889102e043e9b0d12bc8bfb8ecb2: Deleting consensus metadata
I20260808 22:35:33.695036 27807 ts_tablet_manager.cc:1939] T 7b62be4a374a45ee816888d0800b4890 P 649d19608c004f209abcc801b62026a1: Deleting consensus metadata
I20260808 22:35:33.696165 27017 catalog_manager.cc:4841] TS baf5889102e043e9b0d12bc8bfb8ecb2 (127.31.87.130:45137): tablet 7b62be4a374a45ee816888d0800b4890 (table test-table [id=5cf1c348c05d445db48b81280b43ccb3]) successfully deleted
W20260808 22:35:33.696405 27018 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.87.129:35797 (error 108)
I20260808 22:35:33.696471 27017 catalog_manager.cc:4841] TS 649d19608c004f209abcc801b62026a1 (127.31.87.131:39809): tablet 7b62be4a374a45ee816888d0800b4890 (table test-table [id=5cf1c348c05d445db48b81280b43ccb3]) successfully deleted
I20260808 22:35:33.696623 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 27337
I20260808 22:35:33.701653 27756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:33.701368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:35797 (local address 127.0.0.1:44350)
0808 22:35:33.701519 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:33.701578 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.129:35797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
W20260808 22:35:33.701772 27018 catalog_manager.cc:4584] TS a943a6019fa649c18d04a6ec3db45703 (127.31.87.129:35797): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7b62be4a374a45ee816888d0800b4890: Network error: Client connection negotiation failed: client connection to 127.31.87.129:35797: connect: Connection refused (error 111)
I20260808 22:35:33.713469 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 27532
I20260808 22:35:33.733875 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 26993
2026-08-08T22:35:33Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1030 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:35:33 dist-test-slave-266d krb5kdc[27815](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:35:33 dist-test-slave-266d krb5kdc[27815](info): set up 2 sockets
Aug 08 22:35:33 dist-test-slave-266d krb5kdc[27815](info): commencing operation
krb5kdc: starting...
W20260808 22:35:34.875558 32094 mini_kdc.cc:121] Time spent starting KDC: real 1.118s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Aug 08 22:35:34 dist-test-slave-266d krb5kdc[27815](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228534, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:35:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:35:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.87.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
I20260808 22:35:34.990163 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:38943
--webserver_interface=127.31.87.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:36149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:38943
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:35:35.099699 27831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:35:35.099944 27831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:35:35.099988 27831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:35:35.103941 27831 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:35:35.104001 27831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:35:35.104027 27831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:35:35.104063 27831 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:35:35.104096 27831 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:35:35.109187 27831 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:36149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:38943
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:38943
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.87.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27831
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:35:35.110110 27831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:35:35.111056 27831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:35:35.119318 27841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:35.119457 27831 server_base.cc:1034] running on GCE node
W20260808 22:35:35.119347 27840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:35.119356 27843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:35.120143 27831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:35:35.121234 27831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:35:35.122413 27831 hybrid_clock.cc:648] HybridClock initialized: now 1786228535122392 us; error 31 us; skew 500 ppm
Aug 08 22:35:35 dist-test-slave-266d krb5kdc[27815](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228535, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:35:35.127931 27831 init.cc:377] Logged in from keytab as kudu/127.31.87.190@KRBTEST.COM (short username kudu)
I20260808 22:35:35.128834 27831 webserver.cc:460] Webserver started at http://127.31.87.190:43747/ using document root <none> and password file <none>
I20260808 22:35:35.129411 27831 fs_manager.cc:359] Metadata directory not provided
I20260808 22:35:35.129485 27831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:35:35.129729 27831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:35:35.131819 27831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/master-0/data/instance:
uuid: "e6272adfd6b1400f8b329a5dc0bbc329"
format_stamp: "Formatted at 2026-08-08 22:35:35 on dist-test-slave-266d"
server_key: "00294832cc96993506b7717c0661de36"
server_key_iv: "60298c6a735e555ef49d42af879c210c"
server_key_version: "encryptionkey@0"
I20260808 22:35:35.132364 27831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/master-0/wal/instance:
uuid: "e6272adfd6b1400f8b329a5dc0bbc329"
format_stamp: "Formatted at 2026-08-08 22:35:35 on dist-test-slave-266d"
server_key: "00294832cc96993506b7717c0661de36"
server_key_iv: "60298c6a735e555ef49d42af879c210c"
server_key_version: "encryptionkey@0"
I20260808 22:35:35.136049 27831 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260808 22:35:35.138396 27854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:35.139488 27831 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:35:35.139616 27831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "e6272adfd6b1400f8b329a5dc0bbc329"
format_stamp: "Formatted at 2026-08-08 22:35:35 on dist-test-slave-266d"
server_key: "00294832cc96993506b7717c0661de36"
server_key_iv: "60298c6a735e555ef49d42af879c210c"
server_key_version: "encryptionkey@0"
I20260808 22:35:35.139710 27831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:35:35.151197 27831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:35:35.153738 27831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:35:35.153914 27831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:35:35.160939 27831 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.190:38943
I20260808 22:35:35.160945 27948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.190:38943 every 8 connection(s)
I20260808 22:35:35.161862 27831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:35:35.164383 27949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:35:35.165340 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 27831
I20260808 22:35:35.165479 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/master-0/wal/instance
I20260808 22:35:35.165761 32094 external_mini_cluster.cc:1428] Setting key 2a036218e6bcb31f2c9d5b562c4bf41c
I20260808 22:35:35.170166 27949 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6272adfd6b1400f8b329a5dc0bbc329: Bootstrap starting.
I20260808 22:35:35.172571 27949 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e6272adfd6b1400f8b329a5dc0bbc329: Neither blocks nor log segments found. Creating new log.
I20260808 22:35:35.173424 27949 log.cc:824] T 00000000000000000000000000000000 P e6272adfd6b1400f8b329a5dc0bbc329: Log is configured to *not* fsync() on all Append() calls
Aug 08 22:35:35 dist-test-slave-266d krb5kdc[27815](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228534, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:35:35.175436 27949 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6272adfd6b1400f8b329a5dc0bbc329: No bootstrap required, opened a new log
I20260808 22:35:35.179689 27953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:35.167045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51444 (local address 127.31.87.190:38943)
0808 22:35:35.167452 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:35.167469 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:35.167506 (+ 37us) server_negotiation.cc:407] Connection header received
0808 22:35:35.168141 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:35.168156 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:35.168476 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:35.168762 (+ 286us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:35.169682 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:35.170670 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:35.171806 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:35.172246 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:35.172363 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:35.174930 (+ 2567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:35.174936 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:35.174951 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:35:35.174980 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:35.176819 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:35:35.177217 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:35:35.177223 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:35:35.177229 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:35:35.177365 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:35:35.177599 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:35:35.177602 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:35:35.177604 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:35:35.178119 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:35.178321 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:35.178496 (+ 175us) server_negotiation.cc:299] Negotiation successful
0808 22:35:35.178782 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":117,"threads_started":1}
I20260808 22:35:35.179842 27949 raft_consensus.cc:348] T 00000000000000000000000000000000 P e6272adfd6b1400f8b329a5dc0bbc329 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6272adfd6b1400f8b329a5dc0bbc329" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 38943 } }
I20260808 22:35:35.180085 27949 raft_consensus.cc:374] T 00000000000000000000000000000000 P e6272adfd6b1400f8b329a5dc0bbc329 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:35:35.180145 27949 raft_consensus.cc:729] T 00000000000000000000000000000000 P e6272adfd6b1400f8b329a5dc0bbc329 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6272adfd6b1400f8b329a5dc0bbc329, State: Initialized, Role: FOLLOWER
I20260808 22:35:35.180583 27949 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6272adfd6b1400f8b329a5dc0bbc329 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6272adfd6b1400f8b329a5dc0bbc329" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 38943 } }
I20260808 22:35:35.180751 27949 raft_consensus.cc:388] T 00000000000000000000000000000000 P e6272adfd6b1400f8b329a5dc0bbc329 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:35:35.180815 27949 raft_consensus.cc:482] T 00000000000000000000000000000000 P e6272adfd6b1400f8b329a5dc0bbc329 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:35:35.180907 27949 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e6272adfd6b1400f8b329a5dc0bbc329 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:35:35.181785 27949 raft_consensus.cc:504] T 00000000000000000000000000000000 P e6272adfd6b1400f8b329a5dc0bbc329 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6272adfd6b1400f8b329a5dc0bbc329" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 38943 } }
I20260808 22:35:35.182096 27949 leader_election.cc:302] T 00000000000000000000000000000000 P e6272adfd6b1400f8b329a5dc0bbc329 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e6272adfd6b1400f8b329a5dc0bbc329; no voters:
I20260808 22:35:35.182341 27949 leader_election.cc:288] T 00000000000000000000000000000000 P e6272adfd6b1400f8b329a5dc0bbc329 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:35:35.182433 27955 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e6272adfd6b1400f8b329a5dc0bbc329 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:35:35.182642 27955 raft_consensus.cc:686] T 00000000000000000000000000000000 P e6272adfd6b1400f8b329a5dc0bbc329 [term 1 LEADER]: Becoming Leader. State: Replica: e6272adfd6b1400f8b329a5dc0bbc329, State: Running, Role: LEADER
I20260808 22:35:35.182986 27955 consensus_queue.cc:237] T 00000000000000000000000000000000 P e6272adfd6b1400f8b329a5dc0bbc329 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6272adfd6b1400f8b329a5dc0bbc329" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 38943 } }
I20260808 22:35:35.183406 27949 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6272adfd6b1400f8b329a5dc0bbc329 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:35:35.184430 27957 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6272adfd6b1400f8b329a5dc0bbc329 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6272adfd6b1400f8b329a5dc0bbc329. Latest consensus state: current_term: 1 leader_uuid: "e6272adfd6b1400f8b329a5dc0bbc329" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6272adfd6b1400f8b329a5dc0bbc329" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 38943 } } }
I20260808 22:35:35.184554 27957 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6272adfd6b1400f8b329a5dc0bbc329 [sys.catalog]: This master's current role is: LEADER
I20260808 22:35:35.184937 27967 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:35:35.184900 27956 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6272adfd6b1400f8b329a5dc0bbc329 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e6272adfd6b1400f8b329a5dc0bbc329" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6272adfd6b1400f8b329a5dc0bbc329" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 38943 } } }
I20260808 22:35:35.185883 27956 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6272adfd6b1400f8b329a5dc0bbc329 [sys.catalog]: This master's current role is: LEADER
I20260808 22:35:35.187590 27967 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:35:35.192783 27967 catalog_manager.cc:1347] Generated new cluster ID: b9159c4e53c944248f72b0712823a29a
I20260808 22:35:35.192842 27967 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:35:35.205282 27967 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:35:35.206691 27967 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:35:35.214607 27967 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e6272adfd6b1400f8b329a5dc0bbc329: Generated new TSK 0
I20260808 22:35:35.215459 27967 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.87.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab.
Entry for principal kudu/127.31.87.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab.
I20260808 22:35:35.251816 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.129:0
--local_ip_for_outbound_sockets=127.31.87.129
--webserver_interface=127.31.87.129
--webserver_port=0
--tserver_master_addrs=127.31.87.190:38943
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab
--principal=kudu/127.31.87.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:36149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:35:35.378384 27981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:35:35.378612 27981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:35:35.378652 27981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:35:35.382485 27981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:35:35.382531 27981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:35:35.382596 27981 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.129
I20260808 22:35:35.387729 27981 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:36149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab
--principal=kudu/127.31.87.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.87.129
--webserver_port=0
--tserver_master_addrs=127.31.87.190:38943
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27981
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.129
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:35:35.388623 27981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:35:35.389668 27981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:35:35.398428 27988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:35.398572 27981 server_base.cc:1034] running on GCE node
W20260808 22:35:35.398391 27992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:35.398389 27987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:35.399052 27981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:35:35.399644 27981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:35:35.400792 27981 hybrid_clock.cc:648] HybridClock initialized: now 1786228535400776 us; error 26 us; skew 500 ppm
Aug 08 22:35:35 dist-test-slave-266d krb5kdc[27815](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228535, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:35:35.403270 27981 init.cc:377] Logged in from keytab as kudu/127.31.87.129@KRBTEST.COM (short username kudu)
I20260808 22:35:35.404053 27981 webserver.cc:460] Webserver started at http://127.31.87.129:38201/ using document root <none> and password file <none>
I20260808 22:35:35.404552 27981 fs_manager.cc:359] Metadata directory not provided
I20260808 22:35:35.404613 27981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:35:35.404795 27981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:35:35.406666 27981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/ts-0/data/instance:
uuid: "70506579f4ca49ad8d507163e84b4686"
format_stamp: "Formatted at 2026-08-08 22:35:35 on dist-test-slave-266d"
server_key: "acc0b7a2ac7ffd73ea7f759dc7b80322"
server_key_iv: "9acbdfc32621f19184eab231c594c4fd"
server_key_version: "encryptionkey@0"
I20260808 22:35:35.407171 27981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/ts-0/wal/instance:
uuid: "70506579f4ca49ad8d507163e84b4686"
format_stamp: "Formatted at 2026-08-08 22:35:35 on dist-test-slave-266d"
server_key: "acc0b7a2ac7ffd73ea7f759dc7b80322"
server_key_iv: "9acbdfc32621f19184eab231c594c4fd"
server_key_version: "encryptionkey@0"
I20260808 22:35:35.410496 27981 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:35:35.412581 28002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:35.413702 27981 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:35:35.413810 27981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/ts-0/wal
uuid: "70506579f4ca49ad8d507163e84b4686"
format_stamp: "Formatted at 2026-08-08 22:35:35 on dist-test-slave-266d"
server_key: "acc0b7a2ac7ffd73ea7f759dc7b80322"
server_key_iv: "9acbdfc32621f19184eab231c594c4fd"
server_key_version: "encryptionkey@0"
I20260808 22:35:35.413893 27981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:35:35.428033 27981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:35:35.430451 27981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:35:35.430606 27981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:35:35.431123 27981 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:35:35.431861 27981 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:35:35.431906 27981 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:35.431954 27981 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:35:35.431982 27981 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:35.440050 27981 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.129:45383
I20260808 22:35:35.440076 28183 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.129:45383 every 8 connection(s)
I20260808 22:35:35.440802 27981 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
I20260808 22:35:35.449180 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 27981
I20260808 22:35:35.449296 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/ts-0/wal/instance
Aug 08 22:35:35 dist-test-slave-266d krb5kdc[27815](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228535, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.129@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:35:35.449568 32094 external_mini_cluster.cc:1428] Setting key 86ea9d888655d759c0555fb7ed922908
I20260808 22:35:35.453349 27953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:35.442424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:36521 (local address 127.31.87.190:38943)
0808 22:35:35.442543 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:35.442547 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:35.443292 (+ 745us) server_negotiation.cc:407] Connection header received
0808 22:35:35.444527 (+ 1235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:35.444530 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:35.444585 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:35.444659 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:35.445729 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:35.446255 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:35.447395 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:35.447646 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:35.447734 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:35.450090 (+ 2356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:35.450094 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:35.450097 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:35:35.450125 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:35.451498 (+ 1373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:35:35.451953 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:35:35.451956 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:35:35.451958 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:35:35.452033 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:35:35.452337 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:35:35.452340 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:35:35.452342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:35:35.452508 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:35.452591 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:35.453040 (+ 449us) server_negotiation.cc:299] Negotiation successful
0808 22:35:35.453189 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 22:35:35.454020 28186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:35.442752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:38943 (local address 127.31.87.129:36521)
0808 22:35:35.443155 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:35.443194 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 22:35:35.444327 (+ 1133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:35:35.444830 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:35:35.444836 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:35:35.445158 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:35:35.445588 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:35.445600 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:35.446373 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:35.446376 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:35.447283 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:35.447294 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:35.447776 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:35.447781 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:35.447956 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:35.448471 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:35:35.448487 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:35:35.449926 (+ 1439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:35:35.451615 (+ 1689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:35:35.451621 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:35:35.451632 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:35:35.451859 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:35:35.452119 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:35:35.452122 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:35:35.452124 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:35:35.452250 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:35:35.452611 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:35:35.452617 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:35:35.452827 (+ 210us) client_negotiation.cc:769] Sending connection context
0808 22:35:35.452978 (+ 151us) client_negotiation.cc:240] Negotiation successful
0808 22:35:35.453289 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":101,"threads_started":1}
I20260808 22:35:35.455060 28184 heartbeater.cc:347] Connected to a master server at 127.31.87.190:38943
I20260808 22:35:35.455265 28184 heartbeater.cc:464] Registering TS with master...
I20260808 22:35:35.455674 28184 heartbeater.cc:511] Master 127.31.87.190:38943 requested a full tablet report, sending...
I20260808 22:35:35.457036 27890 ts_manager.cc:194] Registered new tserver with Master: 70506579f4ca49ad8d507163e84b4686 (127.31.87.129:45383)
I20260808 22:35:35.458149 27890 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.129@KRBTEST.COM'} at 127.31.87.129:36521
WARNING: no policy specified for kudu/127.31.87.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab.
Entry for principal kudu/127.31.87.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab.
I20260808 22:35:35.475212 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.130:0
--local_ip_for_outbound_sockets=127.31.87.130
--webserver_interface=127.31.87.130
--webserver_port=0
--tserver_master_addrs=127.31.87.190:38943
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab
--principal=kudu/127.31.87.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:36149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:35:35.582463 28189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:35:35.582693 28189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:35:35.582734 28189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:35:35.586721 28189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:35:35.586771 28189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:35:35.586838 28189 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.130
I20260808 22:35:35.591850 28189 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:36149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab
--principal=kudu/127.31.87.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.87.130
--webserver_port=0
--tserver_master_addrs=127.31.87.190:38943
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28189
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.130
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:35:35.592759 28189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:35:35.593715 28189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 22:35:35.602923 28189 server_base.cc:1034] running on GCE node
W20260808 22:35:35.602864 28201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:35.602850 28198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:35.602849 28199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:35.603462 28189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:35:35.604048 28189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:35:35.605192 28189 hybrid_clock.cc:648] HybridClock initialized: now 1786228535605173 us; error 34 us; skew 500 ppm
Aug 08 22:35:35 dist-test-slave-266d krb5kdc[27815](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228535, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:35:35.607791 28189 init.cc:377] Logged in from keytab as kudu/127.31.87.130@KRBTEST.COM (short username kudu)
I20260808 22:35:35.608609 28189 webserver.cc:460] Webserver started at http://127.31.87.130:33255/ using document root <none> and password file <none>
I20260808 22:35:35.609184 28189 fs_manager.cc:359] Metadata directory not provided
I20260808 22:35:35.609269 28189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:35:35.609500 28189 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:35:35.611963 28189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/ts-1/data/instance:
uuid: "28f3275fbdc44415b6dba71c34ffaef9"
format_stamp: "Formatted at 2026-08-08 22:35:35 on dist-test-slave-266d"
server_key: "35b35b3a76d086fab73976e7141e3825"
server_key_iv: "8bf7f403fe9f40ecebde434a021eecb1"
server_key_version: "encryptionkey@0"
I20260808 22:35:35.612530 28189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/ts-1/wal/instance:
uuid: "28f3275fbdc44415b6dba71c34ffaef9"
format_stamp: "Formatted at 2026-08-08 22:35:35 on dist-test-slave-266d"
server_key: "35b35b3a76d086fab73976e7141e3825"
server_key_iv: "8bf7f403fe9f40ecebde434a021eecb1"
server_key_version: "encryptionkey@0"
I20260808 22:35:35.616003 28189 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 22:35:35.618345 28212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:35.619500 28189 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:35:35.619597 28189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/ts-1/wal
uuid: "28f3275fbdc44415b6dba71c34ffaef9"
format_stamp: "Formatted at 2026-08-08 22:35:35 on dist-test-slave-266d"
server_key: "35b35b3a76d086fab73976e7141e3825"
server_key_iv: "8bf7f403fe9f40ecebde434a021eecb1"
server_key_version: "encryptionkey@0"
I20260808 22:35:35.619700 28189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:35:35.631821 28189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:35:35.634698 28189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:35:35.634851 28189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:35:35.635355 28189 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:35:35.636121 28189 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:35:35.636168 28189 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:35.636210 28189 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:35:35.636240 28189 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:35.644541 28189 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.130:43777
I20260808 22:35:35.644562 28393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.130:43777 every 8 connection(s)
I20260808 22:35:35.645385 28189 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
I20260808 22:35:35.650715 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 28189
I20260808 22:35:35.650827 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/ts-1/wal/instance
I20260808 22:35:35.651093 32094 external_mini_cluster.cc:1428] Setting key 1f9971105cfaacd09d135ccd3e34120f
Aug 08 22:35:35 dist-test-slave-266d krb5kdc[27815](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228535, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.130@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:35:35.658248 27953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:35.646971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:38819 (local address 127.31.87.190:38943)
0808 22:35:35.647095 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:35.647098 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:35.647841 (+ 743us) server_negotiation.cc:407] Connection header received
0808 22:35:35.649072 (+ 1231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:35.649076 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:35.649154 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:35.649229 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:35.650304 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:35.650861 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:35.652049 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:35.652331 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:35.652422 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:35.654667 (+ 2245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:35.654672 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:35.654675 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:35:35.654703 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:35.656143 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:35:35.656714 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:35:35.656717 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:35:35.656718 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:35:35.656795 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:35:35.657169 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:35:35.657171 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:35:35.657172 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:35:35.657335 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:35.657409 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:35.657999 (+ 590us) server_negotiation.cc:299] Negotiation successful
0808 22:35:35.658118 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 22:35:35.659242 28398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:35.647300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:38943 (local address 127.31.87.130:38819)
0808 22:35:35.647707 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:35.647747 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 22:35:35.648870 (+ 1123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:35:35.649396 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:35:35.649402 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:35:35.649720 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:35:35.650162 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:35.650175 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:35.650973 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:35.650976 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:35.651938 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:35.651949 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:35.652470 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:35.652474 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:35.652624 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:35.653134 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:35:35.653152 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:35:35.654508 (+ 1356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:35:35.656254 (+ 1746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:35:35.656259 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:35:35.656275 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:35:35.656591 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:35:35.656878 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:35:35.656880 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:35:35.656882 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:35:35.657044 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:35:35.657430 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:35:35.657436 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:35:35.657752 (+ 316us) client_negotiation.cc:769] Sending connection context
0808 22:35:35.657966 (+ 214us) client_negotiation.cc:240] Negotiation successful
0808 22:35:35.658279 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":103,"threads_started":1}
I20260808 22:35:35.660241 28396 heartbeater.cc:347] Connected to a master server at 127.31.87.190:38943
I20260808 22:35:35.660475 28396 heartbeater.cc:464] Registering TS with master...
I20260808 22:35:35.661006 28396 heartbeater.cc:511] Master 127.31.87.190:38943 requested a full tablet report, sending...
I20260808 22:35:35.662050 27890 ts_manager.cc:194] Registered new tserver with Master: 28f3275fbdc44415b6dba71c34ffaef9 (127.31.87.130:43777)
WARNING: no policy specified for kudu/127.31.87.131@KRBTEST.COM; defaulting to no policy
I20260808 22:35:35.662595 27890 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.130@KRBTEST.COM'} at 127.31.87.130:38819
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab.
Entry for principal kudu/127.31.87.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab.
I20260808 22:35:35.679193 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.131:0
--local_ip_for_outbound_sockets=127.31.87.131
--webserver_interface=127.31.87.131
--webserver_port=0
--tserver_master_addrs=127.31.87.190:38943
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab
--principal=kudu/127.31.87.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:36149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:35:35.785660 28407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:35:35.785883 28407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:35:35.785938 28407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:35:35.789781 28407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:35:35.789832 28407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:35:35.789911 28407 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.131
I20260808 22:35:35.794835 28407 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:36149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab
--principal=kudu/127.31.87.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.87.131
--webserver_port=0
--tserver_master_addrs=127.31.87.190:38943
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28407
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.131
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:35:35.795804 28407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:35:35.796720 28407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:35:35.805837 28415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:35.805850 28421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:35.805903 28407 server_base.cc:1034] running on GCE node
W20260808 22:35:35.805852 28416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:35.806350 28407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:35:35.806919 28407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:35:35.808073 28407 hybrid_clock.cc:648] HybridClock initialized: now 1786228535808054 us; error 31 us; skew 500 ppm
Aug 08 22:35:35 dist-test-slave-266d krb5kdc[27815](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228535, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:35:35.810613 28407 init.cc:377] Logged in from keytab as kudu/127.31.87.131@KRBTEST.COM (short username kudu)
I20260808 22:35:35.811398 28407 webserver.cc:460] Webserver started at http://127.31.87.131:45157/ using document root <none> and password file <none>
I20260808 22:35:35.811883 28407 fs_manager.cc:359] Metadata directory not provided
I20260808 22:35:35.811941 28407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:35:35.812134 28407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:35:35.814056 28407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/ts-2/data/instance:
uuid: "a0cb46c19f0a4f3091923ced326f22b3"
format_stamp: "Formatted at 2026-08-08 22:35:35 on dist-test-slave-266d"
server_key: "cd03495c11caecebcb268817e66060ea"
server_key_iv: "4882fec9c9bc4704f911a3d18e4b10ff"
server_key_version: "encryptionkey@0"
I20260808 22:35:35.814558 28407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/ts-2/wal/instance:
uuid: "a0cb46c19f0a4f3091923ced326f22b3"
format_stamp: "Formatted at 2026-08-08 22:35:35 on dist-test-slave-266d"
server_key: "cd03495c11caecebcb268817e66060ea"
server_key_iv: "4882fec9c9bc4704f911a3d18e4b10ff"
server_key_version: "encryptionkey@0"
I20260808 22:35:35.817943 28407 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:35:35.820061 28431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:35.821224 28407 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:35:35.821319 28407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/ts-2/wal
uuid: "a0cb46c19f0a4f3091923ced326f22b3"
format_stamp: "Formatted at 2026-08-08 22:35:35 on dist-test-slave-266d"
server_key: "cd03495c11caecebcb268817e66060ea"
server_key_iv: "4882fec9c9bc4704f911a3d18e4b10ff"
server_key_version: "encryptionkey@0"
I20260808 22:35:35.821410 28407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:35:35.851809 28407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:35:35.854740 28407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:35:35.854903 28407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:35:35.855444 28407 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:35:35.856211 28407 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:35:35.856256 28407 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:35.856297 28407 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:35:35.856338 28407 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:35.864404 28407 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.131:35213
I20260808 22:35:35.864457 28605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.131:35213 every 8 connection(s)
I20260808 22:35:35.865262 28407 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
Aug 08 22:35:35 dist-test-slave-266d krb5kdc[27815](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228535, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.131@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:35:35.874665 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 28407
I20260808 22:35:35.874773 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/ts-2/wal/instance
I20260808 22:35:35.875015 32094 external_mini_cluster.cc:1428] Setting key e72963763be0c6c1e10ca23dcc4a4ac0
I20260808 22:35:35.877419 27953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:35.866914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:60867 (local address 127.31.87.190:38943)
0808 22:35:35.867033 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:35.867036 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:35.867782 (+ 746us) server_negotiation.cc:407] Connection header received
0808 22:35:35.868986 (+ 1204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:35.868988 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:35.869036 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:35.869145 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:35.870150 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:35.870657 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:35.871674 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:35.871870 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:35.871936 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:35.874080 (+ 2144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:35.874083 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:35.874085 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:35:35.874117 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:35.875387 (+ 1270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:35:35.875915 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:35:35.875918 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:35:35.875920 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:35:35.876010 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:35:35.876362 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:35:35.876366 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:35:35.876368 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:35:35.876542 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:35.876617 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:35.877212 (+ 595us) server_negotiation.cc:299] Negotiation successful
0808 22:35:35.877305 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 22:35:35.878087 28608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:35.867241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:38943 (local address 127.31.87.131:60867)
0808 22:35:35.867643 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:35.867681 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 22:35:35.868805 (+ 1124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:35:35.869287 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:35:35.869293 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:35:35.869594 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:35:35.870038 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:35.870050 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:35.870743 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:35.870745 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:35.871587 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:35.871596 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:35.871970 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:35.871973 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:35.872100 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:35.872531 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:35:35.872545 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:35:35.873934 (+ 1389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:35:35.875495 (+ 1561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:35:35.875499 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:35:35.875509 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:35:35.875821 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:35:35.876089 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:35:35.876092 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:35:35.876094 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:35:35.876260 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:35:35.876637 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:35:35.876642 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:35:35.876923 (+ 281us) client_negotiation.cc:769] Sending connection context
0808 22:35:35.877142 (+ 219us) client_negotiation.cc:240] Negotiation successful
0808 22:35:35.877375 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":98,"threads_started":1}
I20260808 22:35:35.879030 28606 heartbeater.cc:347] Connected to a master server at 127.31.87.190:38943
I20260808 22:35:35.879217 28606 heartbeater.cc:464] Registering TS with master...
I20260808 22:35:35.879607 28606 heartbeater.cc:511] Master 127.31.87.190:38943 requested a full tablet report, sending...
I20260808 22:35:35.880474 27890 ts_manager.cc:194] Registered new tserver with Master: a0cb46c19f0a4f3091923ced326f22b3 (127.31.87.131:35213)
I20260808 22:35:35.880980 27890 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.131@KRBTEST.COM'} at 127.31.87.131:60867
I20260808 22:35:35.888490 32094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:35:35.896552 27953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:35.890350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51448 (local address 127.31.87.190:38943)
0808 22:35:35.890473 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:35.890476 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:35.890629 (+ 153us) server_negotiation.cc:407] Connection header received
0808 22:35:35.890699 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:35.890701 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:35.890735 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:35.890792 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:35.891379 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:35.891849 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:35.892565 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:35.892762 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:35.892825 (+ 63us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:35.894109 (+ 1284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:35.894112 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:35.894113 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:35:35.894134 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:35.895151 (+ 1017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:35:35.895555 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:35:35.895558 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:35:35.895559 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:35:35.895634 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:35:35.895845 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:35:35.895847 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:35:35.895848 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:35:35.896006 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:35.896107 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:35.896348 (+ 241us) server_negotiation.cc:299] Negotiation successful
0808 22:35:35.896416 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 22:35:35.898926 27890 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51448:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:35:35.901682 27890 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:35:35.914374 28626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:35.909648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:45383 (local address 127.0.0.1:45940)
0808 22:35:35.910186 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:35.910198 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:35:35.910276 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:35:35.910907 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:35:35.910910 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:35:35.910928 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:35:35.911010 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:35.911016 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:35.911950 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:35.911954 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:35.912954 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:35.912963 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:35.913892 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:35.913896 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:35.914001 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:35.914038 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 22:35:35.914143 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 22:35:35.914244 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":85,"threads_started":1}
I20260808 22:35:35.914405 28632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:35.910043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45940 (local address 127.31.87.129:45383)
0808 22:35:35.910482 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:35.910496 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:35.910515 (+ 19us) server_negotiation.cc:407] Connection header received
0808 22:35:35.910578 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:35.910582 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:35.910755 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:35.910886 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:35:35.911112 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:35.911822 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:35.913113 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:35.913775 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:35.913877 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:35.914015 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:35.914178 (+ 163us) server_negotiation.cc:299] Negotiation successful
0808 22:35:35.914267 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":141,"threads_started":1}
I20260808 22:35:35.915125 28624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:35.908971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.130:43777 (local address 127.0.0.1:56278)
0808 22:35:35.909694 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:35.909742 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 22:35:35.909927 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:35:35.910670 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:35:35.910675 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:35:35.910694 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:35:35.910846 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:35.910853 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:35.912298 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:35.912302 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:35.913681 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:35.913692 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:35.914748 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:35.914751 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:35.914848 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:35.914870 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 22:35:35.914936 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 22:35:35.915002 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"mutex_wait_us":125,"thread_start_us":72,"threads_started":1}
I20260808 22:35:35.915163 28622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:35.909153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56278 (local address 127.31.87.130:43777)
0808 22:35:35.909583 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:35.909594 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:35.909829 (+ 235us) server_negotiation.cc:407] Connection header received
0808 22:35:35.910100 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:35.910106 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:35.910324 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:35.910488 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:35:35.910978 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:35.912143 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:35.913821 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:35.914644 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:35.914738 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:35.914861 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:35.914961 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 22:35:35.915046 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":128,"threads_started":1}
I20260808 22:35:35.915575 28621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:35.909252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.131:35213 (local address 127.0.0.1:59104)
0808 22:35:35.909724 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:35.909757 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 22:35:35.909894 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:35:35.910737 (+ 843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:35:35.910740 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:35:35.910753 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:35:35.910855 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:35.910860 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:35.912298 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:35.912302 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:35.913681 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:35.913692 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:35.915066 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:35.915070 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:35.915159 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:35.915181 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 22:35:35.915380 (+ 199us) client_negotiation.cc:240] Negotiation successful
0808 22:35:35.915451 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"mutex_wait_us":66,"thread_start_us":96,"threads_started":1}
I20260808 22:35:35.915633 28625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:35.909384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59104 (local address 127.31.87.131:35213)
0808 22:35:35.909768 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:35.909779 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:35.909831 (+ 52us) server_negotiation.cc:407] Connection header received
0808 22:35:35.910424 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:35.910430 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:35.910616 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:35.910761 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:35:35.910978 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:35.912149 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:35.913809 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:35.914788 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:35.914884 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:35.915005 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:35.915413 (+ 408us) server_negotiation.cc:299] Negotiation successful
0808 22:35:35.915509 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":135,"threads_started":1}
I20260808 22:35:35.917678 28083 tablet_service.cc:1467] Processing CreateTablet for tablet d599256475e8420a932b491cebf968f4 (DEFAULT_TABLE table=test-table [id=c965af89f11b4ede8506cbc542830426]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:35:35.917829 28299 tablet_service.cc:1467] Processing CreateTablet for tablet d599256475e8420a932b491cebf968f4 (DEFAULT_TABLE table=test-table [id=c965af89f11b4ede8506cbc542830426]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:35:35.918716 28299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d599256475e8420a932b491cebf968f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:35:35.918385 28500 tablet_service.cc:1467] Processing CreateTablet for tablet d599256475e8420a932b491cebf968f4 (DEFAULT_TABLE table=test-table [id=c965af89f11b4ede8506cbc542830426]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:35:35.918746 28083 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d599256475e8420a932b491cebf968f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:35:35.919235 28500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d599256475e8420a932b491cebf968f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:35:35.925272 28633 tablet_bootstrap.cc:492] T d599256475e8420a932b491cebf968f4 P 28f3275fbdc44415b6dba71c34ffaef9: Bootstrap starting.
I20260808 22:35:35.926173 28634 tablet_bootstrap.cc:492] T d599256475e8420a932b491cebf968f4 P 70506579f4ca49ad8d507163e84b4686: Bootstrap starting.
I20260808 22:35:35.927795 28633 tablet_bootstrap.cc:654] T d599256475e8420a932b491cebf968f4 P 28f3275fbdc44415b6dba71c34ffaef9: Neither blocks nor log segments found. Creating new log.
I20260808 22:35:35.928287 28634 tablet_bootstrap.cc:654] T d599256475e8420a932b491cebf968f4 P 70506579f4ca49ad8d507163e84b4686: Neither blocks nor log segments found. Creating new log.
I20260808 22:35:35.929066 28633 log.cc:824] T d599256475e8420a932b491cebf968f4 P 28f3275fbdc44415b6dba71c34ffaef9: Log is configured to *not* fsync() on all Append() calls
I20260808 22:35:35.929383 28635 tablet_bootstrap.cc:492] T d599256475e8420a932b491cebf968f4 P a0cb46c19f0a4f3091923ced326f22b3: Bootstrap starting.
I20260808 22:35:35.929477 28634 log.cc:824] T d599256475e8420a932b491cebf968f4 P 70506579f4ca49ad8d507163e84b4686: Log is configured to *not* fsync() on all Append() calls
I20260808 22:35:35.930824 28633 tablet_bootstrap.cc:492] T d599256475e8420a932b491cebf968f4 P 28f3275fbdc44415b6dba71c34ffaef9: No bootstrap required, opened a new log
I20260808 22:35:35.931006 28633 ts_tablet_manager.cc:1397] T d599256475e8420a932b491cebf968f4 P 28f3275fbdc44415b6dba71c34ffaef9: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 22:35:35.931119 28635 tablet_bootstrap.cc:654] T d599256475e8420a932b491cebf968f4 P a0cb46c19f0a4f3091923ced326f22b3: Neither blocks nor log segments found. Creating new log.
I20260808 22:35:35.932229 28635 log.cc:824] T d599256475e8420a932b491cebf968f4 P a0cb46c19f0a4f3091923ced326f22b3: Log is configured to *not* fsync() on all Append() calls
I20260808 22:35:35.932687 28634 tablet_bootstrap.cc:492] T d599256475e8420a932b491cebf968f4 P 70506579f4ca49ad8d507163e84b4686: No bootstrap required, opened a new log
I20260808 22:35:35.932866 28634 ts_tablet_manager.cc:1397] T d599256475e8420a932b491cebf968f4 P 70506579f4ca49ad8d507163e84b4686: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260808 22:35:35.934127 28635 tablet_bootstrap.cc:492] T d599256475e8420a932b491cebf968f4 P a0cb46c19f0a4f3091923ced326f22b3: No bootstrap required, opened a new log
I20260808 22:35:35.934311 28635 ts_tablet_manager.cc:1397] T d599256475e8420a932b491cebf968f4 P a0cb46c19f0a4f3091923ced326f22b3: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 22:35:35.934173 28633 raft_consensus.cc:348] T d599256475e8420a932b491cebf968f4 P 28f3275fbdc44415b6dba71c34ffaef9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28f3275fbdc44415b6dba71c34ffaef9" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 43777 } } peers { permanent_uuid: "a0cb46c19f0a4f3091923ced326f22b3" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 35213 } } peers { permanent_uuid: "70506579f4ca49ad8d507163e84b4686" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 45383 } }
I20260808 22:35:35.934392 28633 raft_consensus.cc:374] T d599256475e8420a932b491cebf968f4 P 28f3275fbdc44415b6dba71c34ffaef9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:35:35.934445 28633 raft_consensus.cc:729] T d599256475e8420a932b491cebf968f4 P 28f3275fbdc44415b6dba71c34ffaef9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28f3275fbdc44415b6dba71c34ffaef9, State: Initialized, Role: FOLLOWER
I20260808 22:35:35.934916 28633 consensus_queue.cc:260] T d599256475e8420a932b491cebf968f4 P 28f3275fbdc44415b6dba71c34ffaef9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28f3275fbdc44415b6dba71c34ffaef9" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 43777 } } peers { permanent_uuid: "a0cb46c19f0a4f3091923ced326f22b3" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 35213 } } peers { permanent_uuid: "70506579f4ca49ad8d507163e84b4686" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 45383 } }
I20260808 22:35:35.935684 28633 ts_tablet_manager.cc:1428] T d599256475e8420a932b491cebf968f4 P 28f3275fbdc44415b6dba71c34ffaef9: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260808 22:35:35.936002 28396 heartbeater.cc:503] Master 127.31.87.190:38943 was elected leader, sending a full tablet report...
I20260808 22:35:35.936458 28634 raft_consensus.cc:348] T d599256475e8420a932b491cebf968f4 P 70506579f4ca49ad8d507163e84b4686 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28f3275fbdc44415b6dba71c34ffaef9" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 43777 } } peers { permanent_uuid: "a0cb46c19f0a4f3091923ced326f22b3" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 35213 } } peers { permanent_uuid: "70506579f4ca49ad8d507163e84b4686" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 45383 } }
I20260808 22:35:35.936678 28634 raft_consensus.cc:374] T d599256475e8420a932b491cebf968f4 P 70506579f4ca49ad8d507163e84b4686 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:35:35.936733 28634 raft_consensus.cc:729] T d599256475e8420a932b491cebf968f4 P 70506579f4ca49ad8d507163e84b4686 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70506579f4ca49ad8d507163e84b4686, State: Initialized, Role: FOLLOWER
I20260808 22:35:35.937165 28634 consensus_queue.cc:260] T d599256475e8420a932b491cebf968f4 P 70506579f4ca49ad8d507163e84b4686 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28f3275fbdc44415b6dba71c34ffaef9" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 43777 } } peers { permanent_uuid: "a0cb46c19f0a4f3091923ced326f22b3" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 35213 } } peers { permanent_uuid: "70506579f4ca49ad8d507163e84b4686" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 45383 } }
I20260808 22:35:35.938091 28634 ts_tablet_manager.cc:1428] T d599256475e8420a932b491cebf968f4 P 70506579f4ca49ad8d507163e84b4686: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 22:35:35.937979 28635 raft_consensus.cc:348] T d599256475e8420a932b491cebf968f4 P a0cb46c19f0a4f3091923ced326f22b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28f3275fbdc44415b6dba71c34ffaef9" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 43777 } } peers { permanent_uuid: "a0cb46c19f0a4f3091923ced326f22b3" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 35213 } } peers { permanent_uuid: "70506579f4ca49ad8d507163e84b4686" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 45383 } }
I20260808 22:35:35.938194 28635 raft_consensus.cc:374] T d599256475e8420a932b491cebf968f4 P a0cb46c19f0a4f3091923ced326f22b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:35:35.938236 28635 raft_consensus.cc:729] T d599256475e8420a932b491cebf968f4 P a0cb46c19f0a4f3091923ced326f22b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0cb46c19f0a4f3091923ced326f22b3, State: Initialized, Role: FOLLOWER
I20260808 22:35:35.938653 28184 heartbeater.cc:503] Master 127.31.87.190:38943 was elected leader, sending a full tablet report...
I20260808 22:35:35.938824 28635 consensus_queue.cc:260] T d599256475e8420a932b491cebf968f4 P a0cb46c19f0a4f3091923ced326f22b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28f3275fbdc44415b6dba71c34ffaef9" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 43777 } } peers { permanent_uuid: "a0cb46c19f0a4f3091923ced326f22b3" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 35213 } } peers { permanent_uuid: "70506579f4ca49ad8d507163e84b4686" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 45383 } }
I20260808 22:35:35.939422 28606 heartbeater.cc:503] Master 127.31.87.190:38943 was elected leader, sending a full tablet report...
I20260808 22:35:35.939702 28635 ts_tablet_manager.cc:1428] T d599256475e8420a932b491cebf968f4 P a0cb46c19f0a4f3091923ced326f22b3: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260808 22:35:35.941789 28185 tablet.cc:2369] T d599256475e8420a932b491cebf968f4 P 70506579f4ca49ad8d507163e84b4686: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:35:35.992830 28643 raft_consensus.cc:482] T d599256475e8420a932b491cebf968f4 P 70506579f4ca49ad8d507163e84b4686 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:35:35.992995 28643 raft_consensus.cc:504] T d599256475e8420a932b491cebf968f4 P 70506579f4ca49ad8d507163e84b4686 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28f3275fbdc44415b6dba71c34ffaef9" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 43777 } } peers { permanent_uuid: "a0cb46c19f0a4f3091923ced326f22b3" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 35213 } } peers { permanent_uuid: "70506579f4ca49ad8d507163e84b4686" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 45383 } }
I20260808 22:35:35.994058 28643 leader_election.cc:288] T d599256475e8420a932b491cebf968f4 P 70506579f4ca49ad8d507163e84b4686 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 28f3275fbdc44415b6dba71c34ffaef9 (127.31.87.130:43777), a0cb46c19f0a4f3091923ced326f22b3 (127.31.87.131:35213)
I20260808 22:35:35.994485 28642 raft_consensus.cc:482] T d599256475e8420a932b491cebf968f4 P 28f3275fbdc44415b6dba71c34ffaef9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:35:35.994660 28642 raft_consensus.cc:504] T d599256475e8420a932b491cebf968f4 P 28f3275fbdc44415b6dba71c34ffaef9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28f3275fbdc44415b6dba71c34ffaef9" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 43777 } } peers { permanent_uuid: "a0cb46c19f0a4f3091923ced326f22b3" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 35213 } } peers { permanent_uuid: "70506579f4ca49ad8d507163e84b4686" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 45383 } }
I20260808 22:35:35.995942 28642 leader_election.cc:288] T d599256475e8420a932b491cebf968f4 P 28f3275fbdc44415b6dba71c34ffaef9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0cb46c19f0a4f3091923ced326f22b3 (127.31.87.131:35213), 70506579f4ca49ad8d507163e84b4686 (127.31.87.129:45383)
I20260808 22:35:35.999029 28622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:35.994364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:60619 (local address 127.31.87.130:43777)
0808 22:35:35.994526 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:35.994530 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:35.994550 (+ 20us) server_negotiation.cc:407] Connection header received
0808 22:35:35.994792 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:35.994796 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:35.994846 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:35.994895 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:35:35.995345 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:35.996086 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:35.997692 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:35.998336 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:35.998431 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:35.998481 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:35.998786 (+ 305us) server_negotiation.cc:299] Negotiation successful
0808 22:35:35.998904 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":84}
I20260808 22:35:35.999418 28647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:35.994235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.130:43777 (local address 127.31.87.129:60619)
0808 22:35:35.994486 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:35.994501 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:35:35.994645 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:35:35.995081 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:35:35.995084 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:35:35.995105 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:35:35.995222 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:35.995228 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:35.996249 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:35.996252 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:35.997559 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:35.997570 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:35.998540 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:35.998544 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:35.998644 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:35.998671 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 22:35:35.999201 (+ 530us) client_negotiation.cc:240] Negotiation successful
0808 22:35:35.999276 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":88,"threads_started":1}
I20260808 22:35:35.999647 28625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:35.994723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:55219 (local address 127.31.87.131:35213)
0808 22:35:35.994834 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:35.994838 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:35.994958 (+ 120us) server_negotiation.cc:407] Connection header received
0808 22:35:35.995132 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:35.995136 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:35.995185 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:35.995261 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:35:35.995542 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:35.996225 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:35.998331 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:35.998909 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:35.999012 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:35.999056 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:35.999457 (+ 401us) server_negotiation.cc:299] Negotiation successful
0808 22:35:35.999532 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 22:35:36.000133 28648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:35.994305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.131:35213 (local address 127.31.87.129:55219)
0808 22:35:35.994858 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:35.994873 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:35:35.994979 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:35:35.995284 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:35:35.995287 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:35:35.995306 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:35:35.995424 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:35.995431 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:35.996378 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:35.996382 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:35.998180 (+ 1798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:35.998188 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:35.999205 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:35.999209 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:35.999317 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:35.999340 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 22:35:35.999885 (+ 545us) client_negotiation.cc:240] Negotiation successful
0808 22:35:35.999985 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"mutex_wait_us":49,"thread_start_us":119,"threads_started":1}
I20260808 22:35:36.000173 28325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d599256475e8420a932b491cebf968f4" candidate_uuid: "70506579f4ca49ad8d507163e84b4686" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28f3275fbdc44415b6dba71c34ffaef9" is_pre_election: true
I20260808 22:35:36.000373 28325 raft_consensus.cc:2454] T d599256475e8420a932b491cebf968f4 P 28f3275fbdc44415b6dba71c34ffaef9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70506579f4ca49ad8d507163e84b4686 in term 0.
I20260808 22:35:36.000634 28398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:35.995919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.131:35213 (local address 127.31.87.130:34121)
0808 22:35:35.996324 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:35.996338 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:35:35.996411 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:35:35.997202 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:35:35.997205 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:35:35.997222 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:35:35.997308 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:35.997313 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:35.998323 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:35.998335 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:35.999403 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:35.999414 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:36.000290 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:36.000294 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:36.000390 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:36.000414 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 22:35:36.000463 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 22:35:36.000527 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":35}
I20260808 22:35:36.000808 28529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d599256475e8420a932b491cebf968f4" candidate_uuid: "70506579f4ca49ad8d507163e84b4686" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0cb46c19f0a4f3091923ced326f22b3" is_pre_election: true
I20260808 22:35:36.000991 28632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:35.996118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:39067 (local address 127.31.87.129:45383)
0808 22:35:35.996230 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:35.996234 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:35.996475 (+ 241us) server_negotiation.cc:407] Connection header received
0808 22:35:35.996631 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:35.996634 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:35.996676 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:35.996721 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:35:35.997138 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:35.997798 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:35.999624 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:36.000161 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:36.000244 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:36.000287 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:36.000825 (+ 538us) server_negotiation.cc:299] Negotiation successful
0808 22:35:36.000894 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":16,"server-negotiator.queue_time_us":53}
I20260808 22:35:36.000960 28006 leader_election.cc:302] T d599256475e8420a932b491cebf968f4 P 70506579f4ca49ad8d507163e84b4686 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28f3275fbdc44415b6dba71c34ffaef9, 70506579f4ca49ad8d507163e84b4686; no voters:
I20260808 22:35:36.001052 28652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:35.996631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:34121 (local address 127.31.87.131:35213)
0808 22:35:35.996947 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:35.996960 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:35.996971 (+ 11us) server_negotiation.cc:407] Connection header received
0808 22:35:35.997010 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:35.997012 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:35.997051 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:35.997389 (+ 338us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:35:35.997463 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:35.998200 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:35.999578 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:36.000157 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:36.000802 (+ 645us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:36.000845 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:36.000879 (+ 34us) server_negotiation.cc:299] Negotiation successful
0808 22:35:36.000934 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":95,"threads_started":1}
I20260808 22:35:36.001067 28529 raft_consensus.cc:2454] T d599256475e8420a932b491cebf968f4 P a0cb46c19f0a4f3091923ced326f22b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70506579f4ca49ad8d507163e84b4686 in term 0.
I20260808 22:35:36.001241 28643 raft_consensus.cc:2781] T d599256475e8420a932b491cebf968f4 P 70506579f4ca49ad8d507163e84b4686 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:35:36.001332 28643 raft_consensus.cc:482] T d599256475e8420a932b491cebf968f4 P 70506579f4ca49ad8d507163e84b4686 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:35:36.001389 28643 raft_consensus.cc:3037] T d599256475e8420a932b491cebf968f4 P 70506579f4ca49ad8d507163e84b4686 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:35:36.001469 28651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:35.995919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:45383 (local address 127.31.87.130:39067)
0808 22:35:35.996403 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:35.996414 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:35:35.996528 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:35:35.996845 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:35:35.996848 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:35:35.996866 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:35:35.996975 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:35.996982 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:35.997899 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:35.997902 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:35.999434 (+ 1532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:35.999444 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:36.000499 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:36.000503 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:36.000599 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:36.000621 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 22:35:36.001217 (+ 596us) client_negotiation.cc:240] Negotiation successful
0808 22:35:36.001289 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"mutex_wait_us":278,"thread_start_us":107,"threads_started":1}
I20260808 22:35:36.001549 28529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d599256475e8420a932b491cebf968f4" candidate_uuid: "28f3275fbdc44415b6dba71c34ffaef9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0cb46c19f0a4f3091923ced326f22b3" is_pre_election: true
I20260808 22:35:36.001672 28529 raft_consensus.cc:2454] T d599256475e8420a932b491cebf968f4 P a0cb46c19f0a4f3091923ced326f22b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28f3275fbdc44415b6dba71c34ffaef9 in term 0.
I20260808 22:35:36.002178 28111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d599256475e8420a932b491cebf968f4" candidate_uuid: "28f3275fbdc44415b6dba71c34ffaef9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70506579f4ca49ad8d507163e84b4686" is_pre_election: true
I20260808 22:35:36.002192 28215 leader_election.cc:302] T d599256475e8420a932b491cebf968f4 P 28f3275fbdc44415b6dba71c34ffaef9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28f3275fbdc44415b6dba71c34ffaef9, a0cb46c19f0a4f3091923ced326f22b3; no voters:
I20260808 22:35:36.002447 28642 raft_consensus.cc:2781] T d599256475e8420a932b491cebf968f4 P 28f3275fbdc44415b6dba71c34ffaef9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:35:36.002532 28642 raft_consensus.cc:482] T d599256475e8420a932b491cebf968f4 P 28f3275fbdc44415b6dba71c34ffaef9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:35:36.002588 28642 raft_consensus.cc:3037] T d599256475e8420a932b491cebf968f4 P 28f3275fbdc44415b6dba71c34ffaef9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:35:36.002744 28643 raft_consensus.cc:504] T d599256475e8420a932b491cebf968f4 P 70506579f4ca49ad8d507163e84b4686 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28f3275fbdc44415b6dba71c34ffaef9" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 43777 } } peers { permanent_uuid: "a0cb46c19f0a4f3091923ced326f22b3" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 35213 } } peers { permanent_uuid: "70506579f4ca49ad8d507163e84b4686" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 45383 } }
I20260808 22:35:36.003003 28111 raft_consensus.cc:2379] T d599256475e8420a932b491cebf968f4 P 70506579f4ca49ad8d507163e84b4686 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28f3275fbdc44415b6dba71c34ffaef9 in current term 1: Already voted for candidate 70506579f4ca49ad8d507163e84b4686 in this term.
I20260808 22:35:36.003122 28643 leader_election.cc:288] T d599256475e8420a932b491cebf968f4 P 70506579f4ca49ad8d507163e84b4686 [CANDIDATE]: Term 1 election: Requested vote from peers 28f3275fbdc44415b6dba71c34ffaef9 (127.31.87.130:43777), a0cb46c19f0a4f3091923ced326f22b3 (127.31.87.131:35213)
I20260808 22:35:36.003779 28642 raft_consensus.cc:504] T d599256475e8420a932b491cebf968f4 P 28f3275fbdc44415b6dba71c34ffaef9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28f3275fbdc44415b6dba71c34ffaef9" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 43777 } } peers { permanent_uuid: "a0cb46c19f0a4f3091923ced326f22b3" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 35213 } } peers { permanent_uuid: "70506579f4ca49ad8d507163e84b4686" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 45383 } }
I20260808 22:35:36.003823 28325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d599256475e8420a932b491cebf968f4" candidate_uuid: "70506579f4ca49ad8d507163e84b4686" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28f3275fbdc44415b6dba71c34ffaef9"
I20260808 22:35:36.004011 28325 raft_consensus.cc:2379] T d599256475e8420a932b491cebf968f4 P 28f3275fbdc44415b6dba71c34ffaef9 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 70506579f4ca49ad8d507163e84b4686 in current term 1: Already voted for candidate 28f3275fbdc44415b6dba71c34ffaef9 in this term.
I20260808 22:35:36.004089 28529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d599256475e8420a932b491cebf968f4" candidate_uuid: "70506579f4ca49ad8d507163e84b4686" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0cb46c19f0a4f3091923ced326f22b3"
I20260808 22:35:36.004127 28642 leader_election.cc:288] T d599256475e8420a932b491cebf968f4 P 28f3275fbdc44415b6dba71c34ffaef9 [CANDIDATE]: Term 1 election: Requested vote from peers a0cb46c19f0a4f3091923ced326f22b3 (127.31.87.131:35213), 70506579f4ca49ad8d507163e84b4686 (127.31.87.129:45383)
I20260808 22:35:36.004191 28529 raft_consensus.cc:3037] T d599256475e8420a932b491cebf968f4 P a0cb46c19f0a4f3091923ced326f22b3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:35:36.004423 28528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d599256475e8420a932b491cebf968f4" candidate_uuid: "28f3275fbdc44415b6dba71c34ffaef9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0cb46c19f0a4f3091923ced326f22b3"
I20260808 22:35:36.004549 28111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d599256475e8420a932b491cebf968f4" candidate_uuid: "28f3275fbdc44415b6dba71c34ffaef9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70506579f4ca49ad8d507163e84b4686"
I20260808 22:35:36.004680 28111 raft_consensus.cc:2379] T d599256475e8420a932b491cebf968f4 P 70506579f4ca49ad8d507163e84b4686 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 28f3275fbdc44415b6dba71c34ffaef9 in current term 1: Already voted for candidate 70506579f4ca49ad8d507163e84b4686 in this term.
I20260808 22:35:36.004971 28214 leader_election.cc:302] T d599256475e8420a932b491cebf968f4 P 28f3275fbdc44415b6dba71c34ffaef9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28f3275fbdc44415b6dba71c34ffaef9; no voters: 70506579f4ca49ad8d507163e84b4686, a0cb46c19f0a4f3091923ced326f22b3
I20260808 22:35:36.005159 28642 raft_consensus.cc:2726] T d599256475e8420a932b491cebf968f4 P 28f3275fbdc44415b6dba71c34ffaef9 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260808 22:35:36.005434 28529 raft_consensus.cc:2454] T d599256475e8420a932b491cebf968f4 P a0cb46c19f0a4f3091923ced326f22b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70506579f4ca49ad8d507163e84b4686 in term 1.
I20260808 22:35:36.005676 28008 leader_election.cc:302] T d599256475e8420a932b491cebf968f4 P 70506579f4ca49ad8d507163e84b4686 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 70506579f4ca49ad8d507163e84b4686, a0cb46c19f0a4f3091923ced326f22b3; no voters: 28f3275fbdc44415b6dba71c34ffaef9
I20260808 22:35:36.005810 28643 raft_consensus.cc:2781] T d599256475e8420a932b491cebf968f4 P 70506579f4ca49ad8d507163e84b4686 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:35:36.006013 28643 raft_consensus.cc:686] T d599256475e8420a932b491cebf968f4 P 70506579f4ca49ad8d507163e84b4686 [term 1 LEADER]: Becoming Leader. State: Replica: 70506579f4ca49ad8d507163e84b4686, State: Running, Role: LEADER
I20260808 22:35:36.006335 28643 consensus_queue.cc:237] T d599256475e8420a932b491cebf968f4 P 70506579f4ca49ad8d507163e84b4686 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28f3275fbdc44415b6dba71c34ffaef9" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 43777 } } peers { permanent_uuid: "a0cb46c19f0a4f3091923ced326f22b3" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 35213 } } peers { permanent_uuid: "70506579f4ca49ad8d507163e84b4686" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 45383 } }
I20260808 22:35:36.009130 27888 catalog_manager.cc:5495] T d599256475e8420a932b491cebf968f4 P 70506579f4ca49ad8d507163e84b4686 reported cstate change: term changed from 0 to 1, leader changed from <none> to 70506579f4ca49ad8d507163e84b4686 (127.31.87.129). New cstate: current_term: 1 leader_uuid: "70506579f4ca49ad8d507163e84b4686" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28f3275fbdc44415b6dba71c34ffaef9" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 43777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0cb46c19f0a4f3091923ced326f22b3" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 35213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70506579f4ca49ad8d507163e84b4686" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 45383 } health_report { overall_health: HEALTHY } } }
I20260808 22:35:36.035185 28632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:36.031993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45950 (local address 127.31.87.129:45383)
0808 22:35:36.032135 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:36.032139 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:36.032153 (+ 14us) server_negotiation.cc:407] Connection header received
0808 22:35:36.032260 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:36.032262 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:36.032315 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:36.032399 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:35:36.032575 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:36.033048 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:36.033880 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:36.034086 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:36.034167 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:36.034386 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:35:36.034773 (+ 387us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:35:36.034903 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:36.035003 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 22:35:36.035036 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 22:35:36.040525 28325 raft_consensus.cc:1261] T d599256475e8420a932b491cebf968f4 P 28f3275fbdc44415b6dba71c34ffaef9 [term 1 FOLLOWER]: Refusing update from remote peer 70506579f4ca49ad8d507163e84b4686: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:35:36.040522 28529 raft_consensus.cc:1261] T d599256475e8420a932b491cebf968f4 P a0cb46c19f0a4f3091923ced326f22b3 [term 1 FOLLOWER]: Refusing update from remote peer 70506579f4ca49ad8d507163e84b4686: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:35:36.041066 28653 consensus_queue.cc:1035] T d599256475e8420a932b491cebf968f4 P 70506579f4ca49ad8d507163e84b4686 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28f3275fbdc44415b6dba71c34ffaef9" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 43777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:35:36.041268 28643 consensus_queue.cc:1035] T d599256475e8420a932b491cebf968f4 P 70506579f4ca49ad8d507163e84b4686 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0cb46c19f0a4f3091923ced326f22b3" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 35213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:35:36.053265 28661 mvcc.cc:205] Tried to move back new op lower bound from 7316392083615768576 to 7316392083484250112. Current Snapshot: MvccSnapshot[applied={T|T < 7316392083615768576 or (T in {7316392083615768576})}]
I20260808 22:35:36.056814 27888 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51448:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:35:36.056946 27888 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51448:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:35:36.059134 27888 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e6272adfd6b1400f8b329a5dc0bbc329: Sending DeleteTablet for 3 replicas of tablet d599256475e8420a932b491cebf968f4
I20260808 22:35:36.059824 28500 tablet_service.cc:1514] Processing DeleteTablet for tablet d599256475e8420a932b491cebf968f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:35:36 UTC) from {username='kudu', principal='kudu/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:59104
I20260808 22:35:36.060088 28083 tablet_service.cc:1514] Processing DeleteTablet for tablet d599256475e8420a932b491cebf968f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:35:36 UTC) from {username='kudu', principal='kudu/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:45940
I20260808 22:35:36.060464 28675 tablet_replica.cc:330] T d599256475e8420a932b491cebf968f4 P 70506579f4ca49ad8d507163e84b4686: stopping tablet replica
I20260808 22:35:36.060492 28299 tablet_service.cc:1514] Processing DeleteTablet for tablet d599256475e8420a932b491cebf968f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:35:36 UTC) from {username='kudu', principal='kudu/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:56278
I20260808 22:35:36.060815 28674 tablet_replica.cc:330] T d599256475e8420a932b491cebf968f4 P a0cb46c19f0a4f3091923ced326f22b3: stopping tablet replica
I20260808 22:35:36.061038 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 27981
I20260808 22:35:36.061131 28674 raft_consensus.cc:2229] T d599256475e8420a932b491cebf968f4 P a0cb46c19f0a4f3091923ced326f22b3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:35:36.072574 28674 raft_consensus.cc:2258] T d599256475e8420a932b491cebf968f4 P a0cb46c19f0a4f3091923ced326f22b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:35:36.073863 28674 ts_tablet_manager.cc:1905] T d599256475e8420a932b491cebf968f4 P a0cb46c19f0a4f3091923ced326f22b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:35:36.076363 28674 ts_tablet_manager.cc:1918] T d599256475e8420a932b491cebf968f4 P a0cb46c19f0a4f3091923ced326f22b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:35:36.076442 28674 log.cc:1196] T d599256475e8420a932b491cebf968f4 P a0cb46c19f0a4f3091923ced326f22b3: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786228291013016-32094-0/minicluster-data/ts-2/wal/wals/d599256475e8420a932b491cebf968f4
I20260808 22:35:36.076697 28674 ts_tablet_manager.cc:1939] T d599256475e8420a932b491cebf968f4 P a0cb46c19f0a4f3091923ced326f22b3: Deleting consensus metadata
I20260808 22:35:36.076697 28676 tablet_replica.cc:330] T d599256475e8420a932b491cebf968f4 P 28f3275fbdc44415b6dba71c34ffaef9: stopping tablet replica
I20260808 22:35:36.077062 28676 raft_consensus.cc:2229] T d599256475e8420a932b491cebf968f4 P 28f3275fbdc44415b6dba71c34ffaef9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:35:36.077395 28676 raft_consensus.cc:2258] T d599256475e8420a932b491cebf968f4 P 28f3275fbdc44415b6dba71c34ffaef9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:35:36.077837 27859 catalog_manager.cc:4841] TS a0cb46c19f0a4f3091923ced326f22b3 (127.31.87.131:35213): tablet d599256475e8420a932b491cebf968f4 (table test-table [id=c965af89f11b4ede8506cbc542830426]) successfully deleted
I20260808 22:35:36.078902 28676 ts_tablet_manager.cc:1905] T d599256475e8420a932b491cebf968f4 P 28f3275fbdc44415b6dba71c34ffaef9: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 22:35:36.079689 27858 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.87.129:45383 (error 108)
I20260808 22:35:36.080241 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 28189
I20260808 22:35:36.082231 28621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:36.080360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:45383 (local address 127.0.0.1:45966)
0808 22:35:36.082099 (+ 1739us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:36.082157 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.129:45383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1653}
W20260808 22:35:36.083333 27858 catalog_manager.cc:4584] TS 70506579f4ca49ad8d507163e84b4686 (127.31.87.129:45383): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d599256475e8420a932b491cebf968f4: Network error: Client connection negotiation failed: client connection to 127.31.87.129:45383: connect: Connection refused (error 111)
I20260808 22:35:36.091104 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 28407
I20260808 22:35:36.091545 28621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:36.091226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.130:43777 (local address 127.0.0.1:56280)
0808 22:35:36.091409 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:36.091458 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.130:43777: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260808 22:35:36.106142 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 27831
2026-08-08T22:35:36Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2374 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-08T22:35:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:35:36Z Disabled control of system clock
I20260808 22:35:36.140321 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:35047
--webserver_interface=127.31.87.190
--webserver_port=0
--builtin_ntp_servers=127.31.87.148:35485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:35047
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 22:35:36.246155 28685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:35:36.246384 28685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:35:36.246418 28685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:35:36.250303 28685 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:35:36.250355 28685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:35:36.250370 28685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:35:36.250398 28685 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:35:36.250424 28685 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:35:36.255012 28685 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:35485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:35047
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:35047
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.87.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:35:36.255946 28685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:35:36.257267 28685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:35:36.265175 28698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:36.265177 28701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:36.265300 28685 server_base.cc:1034] running on GCE node
W20260808 22:35:36.265218 28697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:36.265885 28685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:35:36.266837 28685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:35:36.267987 28685 hybrid_clock.cc:648] HybridClock initialized: now 1786228536267972 us; error 23 us; skew 500 ppm
I20260808 22:35:36.269703 28685 webserver.cc:460] Webserver started at http://127.31.87.190:35831/ using document root <none> and password file <none>
I20260808 22:35:36.270217 28685 fs_manager.cc:359] Metadata directory not provided
I20260808 22:35:36.270274 28685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:35:36.270462 28685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:35:36.272336 28685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/master-0/data/instance:
uuid: "242fd4a93fa04dd5bf5f8b77b937e485"
format_stamp: "Formatted at 2026-08-08 22:35:36 on dist-test-slave-266d"
server_key: "d4a017f1dcf63bd4cafc20c0fec3a1fd"
server_key_iv: "3c460c29687bb798f4f401a846af5668"
server_key_version: "encryptionkey@0"
I20260808 22:35:36.272828 28685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/master-0/wal/instance:
uuid: "242fd4a93fa04dd5bf5f8b77b937e485"
format_stamp: "Formatted at 2026-08-08 22:35:36 on dist-test-slave-266d"
server_key: "d4a017f1dcf63bd4cafc20c0fec3a1fd"
server_key_iv: "3c460c29687bb798f4f401a846af5668"
server_key_version: "encryptionkey@0"
I20260808 22:35:36.276197 28685 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 22:35:36.278375 28708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:36.279449 28685 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:35:36.279553 28685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "242fd4a93fa04dd5bf5f8b77b937e485"
format_stamp: "Formatted at 2026-08-08 22:35:36 on dist-test-slave-266d"
server_key: "d4a017f1dcf63bd4cafc20c0fec3a1fd"
server_key_iv: "3c460c29687bb798f4f401a846af5668"
server_key_version: "encryptionkey@0"
I20260808 22:35:36.279645 28685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:35:36.296824 28685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:35:36.297458 28685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:35:36.297616 28685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:35:36.303918 28685 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.190:35047
I20260808 22:35:36.303937 28793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.190:35047 every 8 connection(s)
I20260808 22:35:36.304749 28685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:35:36.305334 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 28685
I20260808 22:35:36.305446 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/master-0/wal/instance
I20260808 22:35:36.305727 32094 external_mini_cluster.cc:1428] Setting key fe8a3ddbf6dc11fee0d60aead4e98bd7
I20260808 22:35:36.307842 28794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:35:36.313798 28794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 242fd4a93fa04dd5bf5f8b77b937e485: Bootstrap starting.
I20260808 22:35:36.314343 28797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:36.307027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52884 (local address 127.31.87.190:35047)
0808 22:35:36.307614 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:36.307629 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:36.307663 (+ 34us) server_negotiation.cc:407] Connection header received
0808 22:35:36.308297 (+ 634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:36.308314 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:36.308678 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:36.308936 (+ 258us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:36.309288 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:36.310213 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:36.311383 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:36.311810 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:36.311912 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:36.312729 (+ 817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:36.312734 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:36.312745 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:35:36.313084 (+ 339us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:36.313177 (+ 93us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:35:36.313199 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:36.313269 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:36.313422 (+ 153us) server_negotiation.cc:299] Negotiation successful
0808 22:35:36.313666 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":108,"threads_started":1}
I20260808 22:35:36.316025 28794 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 242fd4a93fa04dd5bf5f8b77b937e485: Neither blocks nor log segments found. Creating new log.
I20260808 22:35:36.316787 28794 log.cc:824] T 00000000000000000000000000000000 P 242fd4a93fa04dd5bf5f8b77b937e485: Log is configured to *not* fsync() on all Append() calls
I20260808 22:35:36.318464 28794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 242fd4a93fa04dd5bf5f8b77b937e485: No bootstrap required, opened a new log
I20260808 22:35:36.321856 28794 raft_consensus.cc:348] T 00000000000000000000000000000000 P 242fd4a93fa04dd5bf5f8b77b937e485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "242fd4a93fa04dd5bf5f8b77b937e485" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 35047 } }
I20260808 22:35:36.322050 28794 raft_consensus.cc:374] T 00000000000000000000000000000000 P 242fd4a93fa04dd5bf5f8b77b937e485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:35:36.322099 28794 raft_consensus.cc:729] T 00000000000000000000000000000000 P 242fd4a93fa04dd5bf5f8b77b937e485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242fd4a93fa04dd5bf5f8b77b937e485, State: Initialized, Role: FOLLOWER
I20260808 22:35:36.322463 28794 consensus_queue.cc:260] T 00000000000000000000000000000000 P 242fd4a93fa04dd5bf5f8b77b937e485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "242fd4a93fa04dd5bf5f8b77b937e485" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 35047 } }
I20260808 22:35:36.322579 28794 raft_consensus.cc:388] T 00000000000000000000000000000000 P 242fd4a93fa04dd5bf5f8b77b937e485 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:35:36.322640 28794 raft_consensus.cc:482] T 00000000000000000000000000000000 P 242fd4a93fa04dd5bf5f8b77b937e485 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:35:36.322726 28794 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 242fd4a93fa04dd5bf5f8b77b937e485 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:35:36.323488 28794 raft_consensus.cc:504] T 00000000000000000000000000000000 P 242fd4a93fa04dd5bf5f8b77b937e485 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "242fd4a93fa04dd5bf5f8b77b937e485" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 35047 } }
I20260808 22:35:36.323791 28794 leader_election.cc:302] T 00000000000000000000000000000000 P 242fd4a93fa04dd5bf5f8b77b937e485 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 242fd4a93fa04dd5bf5f8b77b937e485; no voters:
I20260808 22:35:36.324039 28794 leader_election.cc:288] T 00000000000000000000000000000000 P 242fd4a93fa04dd5bf5f8b77b937e485 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:35:36.324131 28801 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 242fd4a93fa04dd5bf5f8b77b937e485 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:35:36.324342 28801 raft_consensus.cc:686] T 00000000000000000000000000000000 P 242fd4a93fa04dd5bf5f8b77b937e485 [term 1 LEADER]: Becoming Leader. State: Replica: 242fd4a93fa04dd5bf5f8b77b937e485, State: Running, Role: LEADER
I20260808 22:35:36.324641 28801 consensus_queue.cc:237] T 00000000000000000000000000000000 P 242fd4a93fa04dd5bf5f8b77b937e485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "242fd4a93fa04dd5bf5f8b77b937e485" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 35047 } }
I20260808 22:35:36.325073 28794 sys_catalog.cc:564] T 00000000000000000000000000000000 P 242fd4a93fa04dd5bf5f8b77b937e485 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:35:36.326121 28806 sys_catalog.cc:455] T 00000000000000000000000000000000 P 242fd4a93fa04dd5bf5f8b77b937e485 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 242fd4a93fa04dd5bf5f8b77b937e485. Latest consensus state: current_term: 1 leader_uuid: "242fd4a93fa04dd5bf5f8b77b937e485" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "242fd4a93fa04dd5bf5f8b77b937e485" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 35047 } } }
I20260808 22:35:36.326267 28806 sys_catalog.cc:458] T 00000000000000000000000000000000 P 242fd4a93fa04dd5bf5f8b77b937e485 [sys.catalog]: This master's current role is: LEADER
I20260808 22:35:36.326123 28805 sys_catalog.cc:455] T 00000000000000000000000000000000 P 242fd4a93fa04dd5bf5f8b77b937e485 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "242fd4a93fa04dd5bf5f8b77b937e485" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "242fd4a93fa04dd5bf5f8b77b937e485" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 35047 } } }
I20260808 22:35:36.326350 28805 sys_catalog.cc:458] T 00000000000000000000000000000000 P 242fd4a93fa04dd5bf5f8b77b937e485 [sys.catalog]: This master's current role is: LEADER
I20260808 22:35:36.326644 28817 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:35:36.329659 28817 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:35:36.335065 28817 catalog_manager.cc:1347] Generated new cluster ID: ef803eb092854ab7b38bf7722bd2bc01
I20260808 22:35:36.335233 28817 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:35:36.343175 28817 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:35:36.343945 28817 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:35:36.348261 28817 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 242fd4a93fa04dd5bf5f8b77b937e485: Generated new TSK 0
I20260808 22:35:36.348789 28817 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:35:36.352531 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.129:0
--local_ip_for_outbound_sockets=127.31.87.129
--webserver_interface=127.31.87.129
--webserver_port=0
--tserver_master_addrs=127.31.87.190:35047
--builtin_ntp_servers=127.31.87.148:35485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 22:35:36.459817 28833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:35:36.460039 28833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:35:36.460074 28833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:35:36.463986 28833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:35:36.464033 28833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:35:36.464095 28833 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.129
I20260808 22:35:36.468676 28833 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:35485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.87.129
--webserver_port=0
--tserver_master_addrs=127.31.87.190:35047
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28833
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.129
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:35:36.469615 28833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:35:36.470875 28833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:35:36.479842 28840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:36.479875 28845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:36.479954 28833 server_base.cc:1034] running on GCE node
W20260808 22:35:36.479897 28841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:36.480445 28833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:35:36.481009 28833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:35:36.482152 28833 hybrid_clock.cc:648] HybridClock initialized: now 1786228536482128 us; error 37 us; skew 500 ppm
I20260808 22:35:36.483808 28833 webserver.cc:460] Webserver started at http://127.31.87.129:37729/ using document root <none> and password file <none>
I20260808 22:35:36.484354 28833 fs_manager.cc:359] Metadata directory not provided
I20260808 22:35:36.484421 28833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:35:36.484618 28833 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:35:36.486534 28833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/ts-0/data/instance:
uuid: "7058cd30e54942b592f4c6db40237e01"
format_stamp: "Formatted at 2026-08-08 22:35:36 on dist-test-slave-266d"
server_key: "0390f941e26671d0ad4c5675dd8af3c0"
server_key_iv: "9c62c9370d114f7124742dbbd8c93759"
server_key_version: "encryptionkey@0"
I20260808 22:35:36.487042 28833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/ts-0/wal/instance:
uuid: "7058cd30e54942b592f4c6db40237e01"
format_stamp: "Formatted at 2026-08-08 22:35:36 on dist-test-slave-266d"
server_key: "0390f941e26671d0ad4c5675dd8af3c0"
server_key_iv: "9c62c9370d114f7124742dbbd8c93759"
server_key_version: "encryptionkey@0"
I20260808 22:35:36.490339 28833 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260808 22:35:36.492515 28855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:36.493636 28833 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 22:35:36.493736 28833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/ts-0/wal
uuid: "7058cd30e54942b592f4c6db40237e01"
format_stamp: "Formatted at 2026-08-08 22:35:36 on dist-test-slave-266d"
server_key: "0390f941e26671d0ad4c5675dd8af3c0"
server_key_iv: "9c62c9370d114f7124742dbbd8c93759"
server_key_version: "encryptionkey@0"
I20260808 22:35:36.493826 28833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:35:36.512038 28833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:35:36.512571 28833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:35:36.512719 28833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:35:36.513243 28833 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:35:36.514065 28833 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:35:36.514109 28833 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:36.514158 28833 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:35:36.514186 28833 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:36.522212 28833 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.129:40341
I20260808 22:35:36.522312 29047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.129:40341 every 8 connection(s)
I20260808 22:35:36.523031 28833 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
I20260808 22:35:36.528990 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 28833
I20260808 22:35:36.529145 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/ts-0/wal/instance
I20260808 22:35:36.529475 32094 external_mini_cluster.cc:1428] Setting key 29bad36bc84c5bfa87667c5ff7a0d9ea
I20260808 22:35:36.531499 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.130:0
--local_ip_for_outbound_sockets=127.31.87.130
--webserver_interface=127.31.87.130
--webserver_port=0
--tserver_master_addrs=127.31.87.190:35047
--builtin_ntp_servers=127.31.87.148:35485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 22:35:36.531960 28797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:36.524685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:60909 (local address 127.31.87.190:35047)
0808 22:35:36.524806 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:36.524809 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:36.525875 (+ 1066us) server_negotiation.cc:407] Connection header received
0808 22:35:36.527195 (+ 1320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:36.527199 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:36.527250 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:36.527327 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:36.528097 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:36.528626 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:36.530084 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:36.530362 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:36.530464 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:36.531488 (+ 1024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:36.531491 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:36.531493 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:35:36.531505 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:36.531525 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:35:36.531534 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:36.531606 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:36.531758 (+ 152us) server_negotiation.cc:299] Negotiation successful
0808 22:35:36.531829 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 22:35:36.532888 29052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:36.525050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:35047 (local address 127.31.87.129:60909)
0808 22:35:36.525743 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:36.525782 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 22:35:36.527014 (+ 1232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:35:36.527465 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:35:36.527471 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:35:36.527549 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:35:36.527991 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:36.528003 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:36.528711 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:36.528731 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:36.529968 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:36.529979 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:36.530499 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:36.530502 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:36.530645 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:36.531213 (+ 568us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:35:36.531215 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:35:36.531266 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:35:36.531343 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:35:36.531618 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:35:36.531629 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:35:36.531636 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 22:35:36.531755 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 22:35:36.532007 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":114,"threads_started":1}
I20260808 22:35:36.534415 29050 heartbeater.cc:347] Connected to a master server at 127.31.87.190:35047
I20260808 22:35:36.534752 29050 heartbeater.cc:464] Registering TS with master...
I20260808 22:35:36.535668 29050 heartbeater.cc:511] Master 127.31.87.190:35047 requested a full tablet report, sending...
I20260808 22:35:36.537451 28736 ts_manager.cc:194] Registered new tserver with Master: 7058cd30e54942b592f4c6db40237e01 (127.31.87.129:40341)
I20260808 22:35:36.538976 28736 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.87.129:60909
W20260808 22:35:36.643440 29054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:35:36.643661 29054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:35:36.643697 29054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:35:36.647548 29054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:35:36.647593 29054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:35:36.647652 29054 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.130
I20260808 22:35:36.652164 29054 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:35485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.87.130
--webserver_port=0
--tserver_master_addrs=127.31.87.190:35047
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29054
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.130
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:35:36.653152 29054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:35:36.654439 29054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:35:36.663787 29063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:36.663903 29054 server_base.cc:1034] running on GCE node
W20260808 22:35:36.663796 29064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:36.663796 29069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:36.664423 29054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:35:36.665010 29054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:35:36.666179 29054 hybrid_clock.cc:648] HybridClock initialized: now 1786228536666160 us; error 29 us; skew 500 ppm
I20260808 22:35:36.667924 29054 webserver.cc:460] Webserver started at http://127.31.87.130:36727/ using document root <none> and password file <none>
I20260808 22:35:36.668464 29054 fs_manager.cc:359] Metadata directory not provided
I20260808 22:35:36.668524 29054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:35:36.668727 29054 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:35:36.670753 29054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/ts-1/data/instance:
uuid: "867a34fae98e4d09a517d8816a63aff0"
format_stamp: "Formatted at 2026-08-08 22:35:36 on dist-test-slave-266d"
server_key: "3bc92ddd444a47c2ca0d13ce01a86a5c"
server_key_iv: "50c2257adc177fb061f73a58accf0a85"
server_key_version: "encryptionkey@0"
I20260808 22:35:36.671278 29054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/ts-1/wal/instance:
uuid: "867a34fae98e4d09a517d8816a63aff0"
format_stamp: "Formatted at 2026-08-08 22:35:36 on dist-test-slave-266d"
server_key: "3bc92ddd444a47c2ca0d13ce01a86a5c"
server_key_iv: "50c2257adc177fb061f73a58accf0a85"
server_key_version: "encryptionkey@0"
I20260808 22:35:36.674927 29054 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20260808 22:35:36.677380 29079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:36.678532 29054 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 22:35:36.678639 29054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/ts-1/wal
uuid: "867a34fae98e4d09a517d8816a63aff0"
format_stamp: "Formatted at 2026-08-08 22:35:36 on dist-test-slave-266d"
server_key: "3bc92ddd444a47c2ca0d13ce01a86a5c"
server_key_iv: "50c2257adc177fb061f73a58accf0a85"
server_key_version: "encryptionkey@0"
I20260808 22:35:36.678735 29054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:35:36.690349 29054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:35:36.690927 29054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:35:36.691080 29054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:35:36.691602 29054 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:35:36.692471 29054 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:35:36.692515 29054 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:36.692564 29054 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:35:36.692590 29054 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:36.700698 29054 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.130:45523
I20260808 22:35:36.700752 29249 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.130:45523 every 8 connection(s)
I20260808 22:35:36.701586 29054 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
I20260808 22:35:36.708570 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 29054
I20260808 22:35:36.708714 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/ts-1/wal/instance
I20260808 22:35:36.709054 32094 external_mini_cluster.cc:1428] Setting key 11e307f76e606de8e02739e42b824076
I20260808 22:35:36.710137 28797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:36.703260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:60529 (local address 127.31.87.190:35047)
0808 22:35:36.703420 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:36.703424 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:36.704414 (+ 990us) server_negotiation.cc:407] Connection header received
0808 22:35:36.705745 (+ 1331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:36.705748 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:36.705797 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:36.705868 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:36.706657 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:36.707148 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:36.708164 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:36.708368 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:36.708433 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:36.709632 (+ 1199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:36.709637 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:36.709640 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:35:36.709654 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:36.709677 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:35:36.709685 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:36.709741 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:36.709929 (+ 188us) server_negotiation.cc:299] Negotiation successful
0808 22:35:36.709991 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 22:35:36.711019 29253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:36.703646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:35047 (local address 127.31.87.130:60529)
0808 22:35:36.704282 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:36.704320 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 22:35:36.705574 (+ 1254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:35:36.706007 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:35:36.706013 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:35:36.706091 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:35:36.706553 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:36.706565 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:36.707231 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:36.707233 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:36.708066 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:36.708075 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:36.708508 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:36.708513 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:36.708714 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:36.709357 (+ 643us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:35:36.709359 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:35:36.709408 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:35:36.709486 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:35:36.709770 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:35:36.709779 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:35:36.709785 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 22:35:36.709903 (+ 118us) client_negotiation.cc:240] Negotiation successful
0808 22:35:36.710112 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":115,"threads_started":1}
I20260808 22:35:36.711275 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.131:0
--local_ip_for_outbound_sockets=127.31.87.131
--webserver_interface=127.31.87.131
--webserver_port=0
--tserver_master_addrs=127.31.87.190:35047
--builtin_ntp_servers=127.31.87.148:35485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 22:35:36.712180 29250 heartbeater.cc:347] Connected to a master server at 127.31.87.190:35047
I20260808 22:35:36.712388 29250 heartbeater.cc:464] Registering TS with master...
I20260808 22:35:36.712853 29250 heartbeater.cc:511] Master 127.31.87.190:35047 requested a full tablet report, sending...
I20260808 22:35:36.713928 28736 ts_manager.cc:194] Registered new tserver with Master: 867a34fae98e4d09a517d8816a63aff0 (127.31.87.130:45523)
I20260808 22:35:36.714474 28736 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.87.130:60529
W20260808 22:35:36.821878 29257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:35:36.822151 29257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:35:36.822191 29257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:35:36.826283 29257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:35:36.826330 29257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:35:36.826390 29257 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.131
I20260808 22:35:36.831030 29257 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:35485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.87.131
--webserver_port=0
--tserver_master_addrs=127.31.87.190:35047
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29257
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.131
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:35:36.831970 29257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:35:36.833294 29257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:35:36.842406 29268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:36.842409 29273 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:36.842464 29266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:36.842561 29257 server_base.cc:1034] running on GCE node
I20260808 22:35:36.843178 29257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:35:36.843770 29257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:35:36.844933 29257 hybrid_clock.cc:648] HybridClock initialized: now 1786228536844913 us; error 28 us; skew 500 ppm
I20260808 22:35:36.846705 29257 webserver.cc:460] Webserver started at http://127.31.87.131:34839/ using document root <none> and password file <none>
I20260808 22:35:36.847234 29257 fs_manager.cc:359] Metadata directory not provided
I20260808 22:35:36.847295 29257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:35:36.847489 29257 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:35:36.849594 29257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/ts-2/data/instance:
uuid: "c9bc8a51c4bd49129013af456cb02fd1"
format_stamp: "Formatted at 2026-08-08 22:35:36 on dist-test-slave-266d"
server_key: "c78b9fb97f34533f7c786c864f2e96ad"
server_key_iv: "b7bd1e52cf6f0b62b57f6638a7a55ebb"
server_key_version: "encryptionkey@0"
I20260808 22:35:36.850126 29257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/ts-2/wal/instance:
uuid: "c9bc8a51c4bd49129013af456cb02fd1"
format_stamp: "Formatted at 2026-08-08 22:35:36 on dist-test-slave-266d"
server_key: "c78b9fb97f34533f7c786c864f2e96ad"
server_key_iv: "b7bd1e52cf6f0b62b57f6638a7a55ebb"
server_key_version: "encryptionkey@0"
I20260808 22:35:36.853561 29257 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:35:36.855808 29281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:36.856940 29257 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:35:36.857043 29257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/ts-2/wal
uuid: "c9bc8a51c4bd49129013af456cb02fd1"
format_stamp: "Formatted at 2026-08-08 22:35:36 on dist-test-slave-266d"
server_key: "c78b9fb97f34533f7c786c864f2e96ad"
server_key_iv: "b7bd1e52cf6f0b62b57f6638a7a55ebb"
server_key_version: "encryptionkey@0"
I20260808 22:35:36.857162 29257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:35:36.874871 29257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:35:36.875689 29257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:35:36.875876 29257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:35:36.876555 29257 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:35:36.877707 29257 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:35:36.877767 29257 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:36.877831 29257 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:35:36.877861 29257 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:36.887110 29257 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.131:42937
I20260808 22:35:36.887132 29460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.131:42937 every 8 connection(s)
I20260808 22:35:36.887995 29257 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
I20260808 22:35:36.896938 28797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:36.889775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:37087 (local address 127.31.87.190:35047)
0808 22:35:36.889901 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:36.889905 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:36.890963 (+ 1058us) server_negotiation.cc:407] Connection header received
0808 22:35:36.892332 (+ 1369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:36.892336 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:36.892398 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:36.892474 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:36.893434 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:36.893967 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:36.895098 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:36.895317 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:36.895413 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:36.896415 (+ 1002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:36.896418 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:36.896420 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:35:36.896433 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:36.896452 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:35:36.896459 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:36.896517 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:36.896712 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 22:35:36.896778 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 22:35:36.897771 29465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:36.890152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:35047 (local address 127.31.87.131:37087)
0808 22:35:36.890813 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:36.890854 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 22:35:36.892124 (+ 1270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:35:36.892654 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:35:36.892660 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:35:36.892757 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:35:36.893285 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:36.893308 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:36.894086 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:36.894089 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:36.894977 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:36.894987 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:36.895457 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:36.895461 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:36.895623 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:36.896148 (+ 525us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:35:36.896149 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:35:36.896188 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:35:36.896250 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:35:36.896527 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:35:36.896536 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:35:36.896543 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 22:35:36.896710 (+ 167us) client_negotiation.cc:240] Negotiation successful
0808 22:35:36.896985 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":126,"threads_started":1}
I20260808 22:35:36.898201 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 29257
I20260808 22:35:36.898330 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/ts-2/wal/instance
I20260808 22:35:36.898661 32094 external_mini_cluster.cc:1428] Setting key eda1b593551e7915565246ac6504bc87
I20260808 22:35:36.898913 29463 heartbeater.cc:347] Connected to a master server at 127.31.87.190:35047
I20260808 22:35:36.899132 29463 heartbeater.cc:464] Registering TS with master...
I20260808 22:35:36.899667 29463 heartbeater.cc:511] Master 127.31.87.190:35047 requested a full tablet report, sending...
I20260808 22:35:36.900628 28736 ts_manager.cc:194] Registered new tserver with Master: c9bc8a51c4bd49129013af456cb02fd1 (127.31.87.131:42937)
I20260808 22:35:36.901211 28736 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.87.131:37087
I20260808 22:35:36.912067 32094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:35:36.917193 28797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:36.913928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52896 (local address 127.31.87.190:35047)
0808 22:35:36.914030 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:36.914034 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:36.914196 (+ 162us) server_negotiation.cc:407] Connection header received
0808 22:35:36.914298 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:36.914300 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:36.914336 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:36.914400 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:36.914608 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:36.915086 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:36.915831 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:36.916033 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:36.916105 (+ 72us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:36.916806 (+ 701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:36.916809 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:36.916811 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:35:36.916822 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:36.916839 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:35:36.916846 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:36.916936 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:36.917018 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 22:35:36.917047 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 22:35:36.919916 28736 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52896:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:35:36.922986 28736 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:35:36.937263 29488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:36.931121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:40341 (local address 127.0.0.1:45434)
0808 22:35:36.932021 (+ 900us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:36.932034 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:35:36.932125 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:35:36.932824 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:35:36.932829 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:35:36.932865 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:35:36.933036 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:36.933044 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:36.934316 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:36.934319 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:36.935702 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:36.935714 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:36.936697 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:36.936701 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:36.936816 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:36.936854 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 22:35:36.936964 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 22:35:36.937081 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":829,"mutex_wait_us":97,"spinlock_wait_cycles":1280,"thread_start_us":91,"threads_started":1}
I20260808 22:35:36.937266 29489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:36.931816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45434 (local address 127.31.87.129:40341)
0808 22:35:36.932255 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:36.932265 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:36.932287 (+ 22us) server_negotiation.cc:407] Connection header received
0808 22:35:36.932364 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:36.932367 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:36.932510 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:36.932660 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:35:36.933354 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:36.934205 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:36.935844 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:36.936578 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:36.936677 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:36.936872 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:36.937003 (+ 131us) server_negotiation.cc:299] Negotiation successful
0808 22:35:36.937122 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":146,"threads_started":1}
I20260808 22:35:36.937439 29483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:36.931795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.131:42937 (local address 127.0.0.1:44756)
0808 22:35:36.933021 (+ 1226us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:36.933034 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:35:36.933140 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:35:36.933552 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:35:36.933555 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:35:36.933575 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:35:36.933656 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:36.933662 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:36.934861 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:36.934864 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:36.935808 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:36.935819 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:36.937027 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:36.937030 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:36.937141 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:36.937166 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 22:35:36.937231 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 22:35:36.937296 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1166,"mutex_wait_us":1,"thread_start_us":90,"threads_started":1}
I20260808 22:35:36.937789 29486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:36.931218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48702 (local address 127.31.87.130:45523)
0808 22:35:36.931593 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:36.931608 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:36.932186 (+ 578us) server_negotiation.cc:407] Connection header received
0808 22:35:36.932287 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:36.932293 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:36.932506 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:36.932656 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:35:36.933317 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:36.934492 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:36.936076 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:36.936837 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:36.936928 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:36.937072 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:36.937548 (+ 476us) server_negotiation.cc:299] Negotiation successful
0808 22:35:36.937643 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":172,"threads_started":1}
I20260808 22:35:36.937820 29490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:36.932320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44756 (local address 127.31.87.131:42937)
0808 22:35:36.932699 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:36.932711 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:36.933140 (+ 429us) server_negotiation.cc:407] Connection header received
0808 22:35:36.933244 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:36.933249 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:36.933418 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:36.933559 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:35:36.933750 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:36.934756 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:36.935929 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:36.936669 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:36.936784 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:36.936962 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:36.937593 (+ 631us) server_negotiation.cc:299] Negotiation successful
0808 22:35:36.937669 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":156,"threads_started":1}
I20260808 22:35:36.937464 29485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:36.931065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.130:45523 (local address 127.0.0.1:48702)
0808 22:35:36.931862 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:36.931911 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 22:35:36.932038 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:35:36.932823 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:35:36.932829 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:35:36.932864 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:35:36.933220 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:36.933226 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:36.934603 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:36.934606 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:36.935961 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:36.935971 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:36.937144 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:36.937147 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:36.937222 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:36.937241 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 22:35:36.937305 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 22:35:36.937362 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"mutex_wait_us":294,"spinlock_wait_cycles":10880,"thread_start_us":95,"threads_started":1}
I20260808 22:35:36.940301 29155 tablet_service.cc:1467] Processing CreateTablet for tablet 4b6020164dde46beafc84e81e93acad5 (DEFAULT_TABLE table=test-table [id=83b8ef07e8aa48d09808ff2100affca8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:35:36.940454 29337 tablet_service.cc:1467] Processing CreateTablet for tablet 4b6020164dde46beafc84e81e93acad5 (DEFAULT_TABLE table=test-table [id=83b8ef07e8aa48d09808ff2100affca8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:35:36.940454 28932 tablet_service.cc:1467] Processing CreateTablet for tablet 4b6020164dde46beafc84e81e93acad5 (DEFAULT_TABLE table=test-table [id=83b8ef07e8aa48d09808ff2100affca8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:35:36.941220 29155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b6020164dde46beafc84e81e93acad5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:35:36.941461 28932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b6020164dde46beafc84e81e93acad5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:35:36.941461 29337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b6020164dde46beafc84e81e93acad5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:35:36.947594 29491 tablet_bootstrap.cc:492] T 4b6020164dde46beafc84e81e93acad5 P 867a34fae98e4d09a517d8816a63aff0: Bootstrap starting.
I20260808 22:35:36.948220 29492 tablet_bootstrap.cc:492] T 4b6020164dde46beafc84e81e93acad5 P c9bc8a51c4bd49129013af456cb02fd1: Bootstrap starting.
I20260808 22:35:36.949410 29493 tablet_bootstrap.cc:492] T 4b6020164dde46beafc84e81e93acad5 P 7058cd30e54942b592f4c6db40237e01: Bootstrap starting.
I20260808 22:35:36.949859 29491 tablet_bootstrap.cc:654] T 4b6020164dde46beafc84e81e93acad5 P 867a34fae98e4d09a517d8816a63aff0: Neither blocks nor log segments found. Creating new log.
I20260808 22:35:36.950350 29492 tablet_bootstrap.cc:654] T 4b6020164dde46beafc84e81e93acad5 P c9bc8a51c4bd49129013af456cb02fd1: Neither blocks nor log segments found. Creating new log.
I20260808 22:35:36.950727 29491 log.cc:824] T 4b6020164dde46beafc84e81e93acad5 P 867a34fae98e4d09a517d8816a63aff0: Log is configured to *not* fsync() on all Append() calls
I20260808 22:35:36.951126 29493 tablet_bootstrap.cc:654] T 4b6020164dde46beafc84e81e93acad5 P 7058cd30e54942b592f4c6db40237e01: Neither blocks nor log segments found. Creating new log.
I20260808 22:35:36.951143 29492 log.cc:824] T 4b6020164dde46beafc84e81e93acad5 P c9bc8a51c4bd49129013af456cb02fd1: Log is configured to *not* fsync() on all Append() calls
I20260808 22:35:36.951828 29493 log.cc:824] T 4b6020164dde46beafc84e81e93acad5 P 7058cd30e54942b592f4c6db40237e01: Log is configured to *not* fsync() on all Append() calls
I20260808 22:35:36.952508 29491 tablet_bootstrap.cc:492] T 4b6020164dde46beafc84e81e93acad5 P 867a34fae98e4d09a517d8816a63aff0: No bootstrap required, opened a new log
I20260808 22:35:36.952688 29491 ts_tablet_manager.cc:1397] T 4b6020164dde46beafc84e81e93acad5 P 867a34fae98e4d09a517d8816a63aff0: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 22:35:36.953609 29493 tablet_bootstrap.cc:492] T 4b6020164dde46beafc84e81e93acad5 P 7058cd30e54942b592f4c6db40237e01: No bootstrap required, opened a new log
I20260808 22:35:36.953753 29493 ts_tablet_manager.cc:1397] T 4b6020164dde46beafc84e81e93acad5 P 7058cd30e54942b592f4c6db40237e01: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 22:35:36.955853 29491 raft_consensus.cc:348] T 4b6020164dde46beafc84e81e93acad5 P 867a34fae98e4d09a517d8816a63aff0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "867a34fae98e4d09a517d8816a63aff0" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 45523 } } peers { permanent_uuid: "7058cd30e54942b592f4c6db40237e01" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40341 } } peers { permanent_uuid: "c9bc8a51c4bd49129013af456cb02fd1" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 42937 } }
I20260808 22:35:36.956053 29491 raft_consensus.cc:374] T 4b6020164dde46beafc84e81e93acad5 P 867a34fae98e4d09a517d8816a63aff0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:35:36.956116 29491 raft_consensus.cc:729] T 4b6020164dde46beafc84e81e93acad5 P 867a34fae98e4d09a517d8816a63aff0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 867a34fae98e4d09a517d8816a63aff0, State: Initialized, Role: FOLLOWER
I20260808 22:35:36.956539 29491 consensus_queue.cc:260] T 4b6020164dde46beafc84e81e93acad5 P 867a34fae98e4d09a517d8816a63aff0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "867a34fae98e4d09a517d8816a63aff0" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 45523 } } peers { permanent_uuid: "7058cd30e54942b592f4c6db40237e01" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40341 } } peers { permanent_uuid: "c9bc8a51c4bd49129013af456cb02fd1" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 42937 } }
I20260808 22:35:36.956749 29493 raft_consensus.cc:348] T 4b6020164dde46beafc84e81e93acad5 P 7058cd30e54942b592f4c6db40237e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "867a34fae98e4d09a517d8816a63aff0" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 45523 } } peers { permanent_uuid: "7058cd30e54942b592f4c6db40237e01" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40341 } } peers { permanent_uuid: "c9bc8a51c4bd49129013af456cb02fd1" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 42937 } }
I20260808 22:35:36.957003 29493 raft_consensus.cc:374] T 4b6020164dde46beafc84e81e93acad5 P 7058cd30e54942b592f4c6db40237e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:35:36.957062 29493 raft_consensus.cc:729] T 4b6020164dde46beafc84e81e93acad5 P 7058cd30e54942b592f4c6db40237e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7058cd30e54942b592f4c6db40237e01, State: Initialized, Role: FOLLOWER
I20260808 22:35:36.957345 29250 heartbeater.cc:503] Master 127.31.87.190:35047 was elected leader, sending a full tablet report...
I20260808 22:35:36.957530 29491 ts_tablet_manager.cc:1428] T 4b6020164dde46beafc84e81e93acad5 P 867a34fae98e4d09a517d8816a63aff0: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 22:35:36.957528 29493 consensus_queue.cc:260] T 4b6020164dde46beafc84e81e93acad5 P 7058cd30e54942b592f4c6db40237e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "867a34fae98e4d09a517d8816a63aff0" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 45523 } } peers { permanent_uuid: "7058cd30e54942b592f4c6db40237e01" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40341 } } peers { permanent_uuid: "c9bc8a51c4bd49129013af456cb02fd1" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 42937 } }
I20260808 22:35:36.957804 29492 tablet_bootstrap.cc:492] T 4b6020164dde46beafc84e81e93acad5 P c9bc8a51c4bd49129013af456cb02fd1: No bootstrap required, opened a new log
I20260808 22:35:36.957981 29492 ts_tablet_manager.cc:1397] T 4b6020164dde46beafc84e81e93acad5 P c9bc8a51c4bd49129013af456cb02fd1: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20260808 22:35:36.958175 29050 heartbeater.cc:503] Master 127.31.87.190:35047 was elected leader, sending a full tablet report...
I20260808 22:35:36.958431 29493 ts_tablet_manager.cc:1428] T 4b6020164dde46beafc84e81e93acad5 P 7058cd30e54942b592f4c6db40237e01: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 22:35:36.961978 29492 raft_consensus.cc:348] T 4b6020164dde46beafc84e81e93acad5 P c9bc8a51c4bd49129013af456cb02fd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "867a34fae98e4d09a517d8816a63aff0" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 45523 } } peers { permanent_uuid: "7058cd30e54942b592f4c6db40237e01" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40341 } } peers { permanent_uuid: "c9bc8a51c4bd49129013af456cb02fd1" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 42937 } }
I20260808 22:35:36.962229 29492 raft_consensus.cc:374] T 4b6020164dde46beafc84e81e93acad5 P c9bc8a51c4bd49129013af456cb02fd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:35:36.962301 29492 raft_consensus.cc:729] T 4b6020164dde46beafc84e81e93acad5 P c9bc8a51c4bd49129013af456cb02fd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9bc8a51c4bd49129013af456cb02fd1, State: Initialized, Role: FOLLOWER
I20260808 22:35:36.962641 29492 consensus_queue.cc:260] T 4b6020164dde46beafc84e81e93acad5 P c9bc8a51c4bd49129013af456cb02fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "867a34fae98e4d09a517d8816a63aff0" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 45523 } } peers { permanent_uuid: "7058cd30e54942b592f4c6db40237e01" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40341 } } peers { permanent_uuid: "c9bc8a51c4bd49129013af456cb02fd1" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 42937 } }
I20260808 22:35:36.963181 29463 heartbeater.cc:503] Master 127.31.87.190:35047 was elected leader, sending a full tablet report...
I20260808 22:35:36.963362 29492 ts_tablet_manager.cc:1428] T 4b6020164dde46beafc84e81e93acad5 P c9bc8a51c4bd49129013af456cb02fd1: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260808 22:35:37.024147 29051 tablet.cc:2369] T 4b6020164dde46beafc84e81e93acad5 P 7058cd30e54942b592f4c6db40237e01: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:35:37.135200 29501 raft_consensus.cc:482] T 4b6020164dde46beafc84e81e93acad5 P c9bc8a51c4bd49129013af456cb02fd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:35:37.135408 29501 raft_consensus.cc:504] T 4b6020164dde46beafc84e81e93acad5 P c9bc8a51c4bd49129013af456cb02fd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "867a34fae98e4d09a517d8816a63aff0" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 45523 } } peers { permanent_uuid: "7058cd30e54942b592f4c6db40237e01" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40341 } } peers { permanent_uuid: "c9bc8a51c4bd49129013af456cb02fd1" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 42937 } }
I20260808 22:35:37.136540 29501 leader_election.cc:288] T 4b6020164dde46beafc84e81e93acad5 P c9bc8a51c4bd49129013af456cb02fd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 867a34fae98e4d09a517d8816a63aff0 (127.31.87.130:45523), 7058cd30e54942b592f4c6db40237e01 (127.31.87.129:40341)
W20260808 22:35:37.138900 29464 tablet.cc:2369] T 4b6020164dde46beafc84e81e93acad5 P c9bc8a51c4bd49129013af456cb02fd1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:35:37.140113 29486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:37.136892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:35353 (local address 127.31.87.130:45523)
0808 22:35:37.137029 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:37.137033 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:37.137055 (+ 22us) server_negotiation.cc:407] Connection header received
0808 22:35:37.137255 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:37.137258 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:37.137304 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:37.137369 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:35:37.137665 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:37.138163 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:37.139214 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:37.139596 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:37.139706 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:37.139758 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:37.139927 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 22:35:37.139997 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 22:35:37.140103 29465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:37.136765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.130:45523 (local address 127.31.87.131:35353)
0808 22:35:37.136985 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:37.137002 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 22:35:37.137124 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:35:37.137396 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:35:37.137399 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:35:37.137418 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:35:37.137545 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:37.137551 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:37.138269 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:37.138272 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:37.139086 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:37.139098 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:37.139713 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:37.139717 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:37.139812 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:37.139832 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 22:35:37.139896 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 22:35:37.139967 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"spinlock_wait_cycles":27392}
I20260808 22:35:37.140587 29502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:37.136774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:40341 (local address 127.31.87.131:51433)
0808 22:35:37.137346 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:37.137357 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:35:37.137473 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:35:37.137700 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:35:37.137703 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:35:37.137718 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:35:37.137837 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:37.137843 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:37.138531 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:37.138535 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:37.139550 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:37.139560 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:37.140163 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:37.140166 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:37.140254 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:37.140287 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 22:35:37.140383 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 22:35:37.140445 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"mutex_wait_us":173,"thread_start_us":159,"threads_started":1}
I20260808 22:35:37.140636 29489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:37.136895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:51433 (local address 127.31.87.129:40341)
0808 22:35:37.137006 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:37.137010 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:37.137449 (+ 439us) server_negotiation.cc:407] Connection header received
0808 22:35:37.137575 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:37.137578 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:37.137625 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:37.137686 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:35:37.137949 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:37.138433 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:37.139672 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:37.140048 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:37.140148 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:37.140195 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:37.140429 (+ 234us) server_negotiation.cc:299] Negotiation successful
0808 22:35:37.140511 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 22:35:37.140770 29185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b6020164dde46beafc84e81e93acad5" candidate_uuid: "c9bc8a51c4bd49129013af456cb02fd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "867a34fae98e4d09a517d8816a63aff0" is_pre_election: true
I20260808 22:35:37.141065 29185 raft_consensus.cc:2454] T 4b6020164dde46beafc84e81e93acad5 P 867a34fae98e4d09a517d8816a63aff0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c9bc8a51c4bd49129013af456cb02fd1 in term 0.
I20260808 22:35:37.141194 28968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b6020164dde46beafc84e81e93acad5" candidate_uuid: "c9bc8a51c4bd49129013af456cb02fd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7058cd30e54942b592f4c6db40237e01" is_pre_election: true
I20260808 22:35:37.141463 28968 raft_consensus.cc:2454] T 4b6020164dde46beafc84e81e93acad5 P 7058cd30e54942b592f4c6db40237e01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c9bc8a51c4bd49129013af456cb02fd1 in term 0.
I20260808 22:35:37.141592 29282 leader_election.cc:302] T 4b6020164dde46beafc84e81e93acad5 P c9bc8a51c4bd49129013af456cb02fd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 867a34fae98e4d09a517d8816a63aff0, c9bc8a51c4bd49129013af456cb02fd1; no voters:
I20260808 22:35:37.141862 29501 raft_consensus.cc:2781] T 4b6020164dde46beafc84e81e93acad5 P c9bc8a51c4bd49129013af456cb02fd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:35:37.141942 29501 raft_consensus.cc:482] T 4b6020164dde46beafc84e81e93acad5 P c9bc8a51c4bd49129013af456cb02fd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:35:37.141973 29501 raft_consensus.cc:3037] T 4b6020164dde46beafc84e81e93acad5 P c9bc8a51c4bd49129013af456cb02fd1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:35:37.143038 29501 raft_consensus.cc:504] T 4b6020164dde46beafc84e81e93acad5 P c9bc8a51c4bd49129013af456cb02fd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "867a34fae98e4d09a517d8816a63aff0" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 45523 } } peers { permanent_uuid: "7058cd30e54942b592f4c6db40237e01" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40341 } } peers { permanent_uuid: "c9bc8a51c4bd49129013af456cb02fd1" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 42937 } }
I20260808 22:35:37.143345 29501 leader_election.cc:288] T 4b6020164dde46beafc84e81e93acad5 P c9bc8a51c4bd49129013af456cb02fd1 [CANDIDATE]: Term 1 election: Requested vote from peers 867a34fae98e4d09a517d8816a63aff0 (127.31.87.130:45523), 7058cd30e54942b592f4c6db40237e01 (127.31.87.129:40341)
I20260808 22:35:37.143647 29185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b6020164dde46beafc84e81e93acad5" candidate_uuid: "c9bc8a51c4bd49129013af456cb02fd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "867a34fae98e4d09a517d8816a63aff0"
I20260808 22:35:37.143661 28968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b6020164dde46beafc84e81e93acad5" candidate_uuid: "c9bc8a51c4bd49129013af456cb02fd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7058cd30e54942b592f4c6db40237e01"
I20260808 22:35:37.143745 28968 raft_consensus.cc:3037] T 4b6020164dde46beafc84e81e93acad5 P 7058cd30e54942b592f4c6db40237e01 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:35:37.143756 29185 raft_consensus.cc:3037] T 4b6020164dde46beafc84e81e93acad5 P 867a34fae98e4d09a517d8816a63aff0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:35:37.144703 28968 raft_consensus.cc:2454] T 4b6020164dde46beafc84e81e93acad5 P 7058cd30e54942b592f4c6db40237e01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9bc8a51c4bd49129013af456cb02fd1 in term 1.
I20260808 22:35:37.144712 29185 raft_consensus.cc:2454] T 4b6020164dde46beafc84e81e93acad5 P 867a34fae98e4d09a517d8816a63aff0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9bc8a51c4bd49129013af456cb02fd1 in term 1.
I20260808 22:35:37.144956 29282 leader_election.cc:302] T 4b6020164dde46beafc84e81e93acad5 P c9bc8a51c4bd49129013af456cb02fd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 867a34fae98e4d09a517d8816a63aff0, c9bc8a51c4bd49129013af456cb02fd1; no voters:
I20260808 22:35:37.145159 29501 raft_consensus.cc:2781] T 4b6020164dde46beafc84e81e93acad5 P c9bc8a51c4bd49129013af456cb02fd1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:35:37.145351 29501 raft_consensus.cc:686] T 4b6020164dde46beafc84e81e93acad5 P c9bc8a51c4bd49129013af456cb02fd1 [term 1 LEADER]: Becoming Leader. State: Replica: c9bc8a51c4bd49129013af456cb02fd1, State: Running, Role: LEADER
I20260808 22:35:37.145669 29501 consensus_queue.cc:237] T 4b6020164dde46beafc84e81e93acad5 P c9bc8a51c4bd49129013af456cb02fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "867a34fae98e4d09a517d8816a63aff0" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 45523 } } peers { permanent_uuid: "7058cd30e54942b592f4c6db40237e01" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40341 } } peers { permanent_uuid: "c9bc8a51c4bd49129013af456cb02fd1" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 42937 } }
I20260808 22:35:37.148530 28735 catalog_manager.cc:5495] T 4b6020164dde46beafc84e81e93acad5 P c9bc8a51c4bd49129013af456cb02fd1 reported cstate change: term changed from 0 to 1, leader changed from <none> to c9bc8a51c4bd49129013af456cb02fd1 (127.31.87.131). New cstate: current_term: 1 leader_uuid: "c9bc8a51c4bd49129013af456cb02fd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "867a34fae98e4d09a517d8816a63aff0" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 45523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7058cd30e54942b592f4c6db40237e01" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40341 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9bc8a51c4bd49129013af456cb02fd1" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 42937 } health_report { overall_health: HEALTHY } } }
I20260808 22:35:37.168771 29490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:37.165971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44772 (local address 127.31.87.131:42937)
0808 22:35:37.166067 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:37.166070 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:37.166081 (+ 11us) server_negotiation.cc:407] Connection header received
0808 22:35:37.166172 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:37.166174 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:37.166219 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:37.166295 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:35:37.166459 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:37.166921 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:37.167688 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:37.167878 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:37.167942 (+ 64us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:37.168162 (+ 220us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:35:37.168510 (+ 348us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:35:37.168618 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:37.168655 (+ 37us) server_negotiation.cc:299] Negotiation successful
0808 22:35:37.168681 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 22:35:37.173736 29185 raft_consensus.cc:1261] T 4b6020164dde46beafc84e81e93acad5 P 867a34fae98e4d09a517d8816a63aff0 [term 1 FOLLOWER]: Refusing update from remote peer c9bc8a51c4bd49129013af456cb02fd1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:35:37.173730 28968 raft_consensus.cc:1261] T 4b6020164dde46beafc84e81e93acad5 P 7058cd30e54942b592f4c6db40237e01 [term 1 FOLLOWER]: Refusing update from remote peer c9bc8a51c4bd49129013af456cb02fd1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:35:37.174371 29503 consensus_queue.cc:1035] T 4b6020164dde46beafc84e81e93acad5 P c9bc8a51c4bd49129013af456cb02fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7058cd30e54942b592f4c6db40237e01" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 40341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:35:37.174530 29501 consensus_queue.cc:1035] T 4b6020164dde46beafc84e81e93acad5 P c9bc8a51c4bd49129013af456cb02fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "867a34fae98e4d09a517d8816a63aff0" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 45523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:35:37.180557 29506 mvcc.cc:205] Tried to move back new op lower bound from 7316392088257724416 to 7316392088150982656. Current Snapshot: MvccSnapshot[applied={T|T < 7316392088257724416}]
I20260808 22:35:37.181553 29507 mvcc.cc:205] Tried to move back new op lower bound from 7316392088257724416 to 7316392088150982656. Current Snapshot: MvccSnapshot[applied={T|T < 7316392088257724416}]
I20260808 22:35:37.188985 28736 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52896:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:35:37.189141 28736 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52896:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:35:37.191311 28736 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 242fd4a93fa04dd5bf5f8b77b937e485: Sending DeleteTablet for 3 replicas of tablet 4b6020164dde46beafc84e81e93acad5
I20260808 22:35:37.192052 29337 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b6020164dde46beafc84e81e93acad5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:35:37 UTC) from {username='slave'} at 127.0.0.1:44756
I20260808 22:35:37.192098 29155 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b6020164dde46beafc84e81e93acad5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:35:37 UTC) from {username='slave'} at 127.0.0.1:48702
I20260808 22:35:37.192098 28932 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b6020164dde46beafc84e81e93acad5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:35:37 UTC) from {username='slave'} at 127.0.0.1:45434
I20260808 22:35:37.192509 29525 tablet_replica.cc:330] T 4b6020164dde46beafc84e81e93acad5 P c9bc8a51c4bd49129013af456cb02fd1: stopping tablet replica
I20260808 22:35:37.192791 29525 raft_consensus.cc:2229] T 4b6020164dde46beafc84e81e93acad5 P c9bc8a51c4bd49129013af456cb02fd1 [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:35:37.193038 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 28833
I20260808 22:35:37.193153 29525 raft_consensus.cc:2258] T 4b6020164dde46beafc84e81e93acad5 P c9bc8a51c4bd49129013af456cb02fd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:35:37.198496 29525 ts_tablet_manager.cc:1905] T 4b6020164dde46beafc84e81e93acad5 P c9bc8a51c4bd49129013af456cb02fd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:35:37.200764 29525 ts_tablet_manager.cc:1918] T 4b6020164dde46beafc84e81e93acad5 P c9bc8a51c4bd49129013af456cb02fd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:35:37.200848 29525 log.cc:1196] T 4b6020164dde46beafc84e81e93acad5 P c9bc8a51c4bd49129013af456cb02fd1: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/ts-2/wal/wals/4b6020164dde46beafc84e81e93acad5
I20260808 22:35:37.201116 29525 ts_tablet_manager.cc:1939] T 4b6020164dde46beafc84e81e93acad5 P c9bc8a51c4bd49129013af456cb02fd1: Deleting consensus metadata
I20260808 22:35:37.202505 28714 catalog_manager.cc:4841] TS c9bc8a51c4bd49129013af456cb02fd1 (127.31.87.131:42937): tablet 4b6020164dde46beafc84e81e93acad5 (table test-table [id=83b8ef07e8aa48d09808ff2100affca8]) successfully deleted
I20260808 22:35:37.204320 29526 tablet_replica.cc:330] T 4b6020164dde46beafc84e81e93acad5 P 867a34fae98e4d09a517d8816a63aff0: stopping tablet replica
I20260808 22:35:37.204576 29526 raft_consensus.cc:2229] T 4b6020164dde46beafc84e81e93acad5 P 867a34fae98e4d09a517d8816a63aff0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:35:37.204852 29526 raft_consensus.cc:2258] T 4b6020164dde46beafc84e81e93acad5 P 867a34fae98e4d09a517d8816a63aff0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:35:37.206091 29526 ts_tablet_manager.cc:1905] T 4b6020164dde46beafc84e81e93acad5 P 867a34fae98e4d09a517d8816a63aff0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:35:37.208550 29526 ts_tablet_manager.cc:1918] T 4b6020164dde46beafc84e81e93acad5 P 867a34fae98e4d09a517d8816a63aff0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:35:37.208639 29526 log.cc:1196] T 4b6020164dde46beafc84e81e93acad5 P 867a34fae98e4d09a517d8816a63aff0: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786228291013016-32094-0/minicluster-data/ts-1/wal/wals/4b6020164dde46beafc84e81e93acad5
I20260808 22:35:37.208940 29526 ts_tablet_manager.cc:1939] T 4b6020164dde46beafc84e81e93acad5 P 867a34fae98e4d09a517d8816a63aff0: Deleting consensus metadata
I20260808 22:35:37.210083 28711 catalog_manager.cc:4841] TS 867a34fae98e4d09a517d8816a63aff0 (127.31.87.130:45523): tablet 4b6020164dde46beafc84e81e93acad5 (table test-table [id=83b8ef07e8aa48d09808ff2100affca8]) successfully deleted
W20260808 22:35:37.211436 28714 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.87.129:40341 (error 108)
I20260808 22:35:37.211593 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 29054
I20260808 22:35:37.214397 29485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:37.214085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:40341 (local address 127.0.0.1:45446)
0808 22:35:37.214244 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:37.214307 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.129:40341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
W20260808 22:35:37.214604 28714 catalog_manager.cc:4584] TS 7058cd30e54942b592f4c6db40237e01 (127.31.87.129:40341): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4b6020164dde46beafc84e81e93acad5: Network error: Client connection negotiation failed: client connection to 127.31.87.129:40341: connect: Connection refused (error 111)
I20260808 22:35:37.219733 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 29257
I20260808 22:35:37.228879 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 28685
2026-08-08T22:35:37Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1124 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:35:37 dist-test-slave-266d krb5kdc[29533](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:35:37 dist-test-slave-266d krb5kdc[29533](info): set up 2 sockets
Aug 08 22:35:37 dist-test-slave-266d krb5kdc[29533](info): commencing operation
krb5kdc: starting...
W20260808 22:35:38.363926 32094 mini_kdc.cc:121] Time spent starting KDC: real 1.107s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Aug 08 22:35:38 dist-test-slave-266d krb5kdc[29533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228538, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:35:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:35:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.87.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
I20260808 22:35:38.479502 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:42029
--webserver_interface=127.31.87.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:44005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:42029
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:35:38.585428 29553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:35:38.585669 29553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:35:38.585707 29553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:35:38.589589 29553 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:35:38.589644 29553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:35:38.589663 29553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:35:38.589677 29553 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:35:38.589690 29553 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:35:38.594666 29553 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:44005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:42029
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:42029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.87.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29553
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:35:38.595570 29553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:35:38.596514 29553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:35:38.604450 29569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:38.604478 29566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:38.604444 29561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:38.604560 29553 server_base.cc:1034] running on GCE node
I20260808 22:35:38.605297 29553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:35:38.606295 29553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:35:38.607456 29553 hybrid_clock.cc:648] HybridClock initialized: now 1786228538607436 us; error 33 us; skew 500 ppm
Aug 08 22:35:38 dist-test-slave-266d krb5kdc[29533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228538, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:35:38.610124 29553 init.cc:377] Logged in from keytab as kudu/127.31.87.190@KRBTEST.COM (short username kudu)
I20260808 22:35:38.610949 29553 webserver.cc:460] Webserver started at http://127.31.87.190:45325/ using document root <none> and password file <none>
I20260808 22:35:38.611464 29553 fs_manager.cc:359] Metadata directory not provided
I20260808 22:35:38.611526 29553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:35:38.611734 29553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:35:38.613698 29553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/master-0/data/instance:
uuid: "589100aff40a4ecbb12e02db3aa96ab1"
format_stamp: "Formatted at 2026-08-08 22:35:38 on dist-test-slave-266d"
server_key: "f5b0b542c1d3a8004afe67a39dc7a98f"
server_key_iv: "67babba5bc0cd403c499d95b48295f6a"
server_key_version: "encryptionkey@0"
I20260808 22:35:38.614219 29553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/master-0/wal/instance:
uuid: "589100aff40a4ecbb12e02db3aa96ab1"
format_stamp: "Formatted at 2026-08-08 22:35:38 on dist-test-slave-266d"
server_key: "f5b0b542c1d3a8004afe67a39dc7a98f"
server_key_iv: "67babba5bc0cd403c499d95b48295f6a"
server_key_version: "encryptionkey@0"
I20260808 22:35:38.617750 29553 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:35:38.620226 29582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:38.621750 29553 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 22:35:38.621871 29553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "589100aff40a4ecbb12e02db3aa96ab1"
format_stamp: "Formatted at 2026-08-08 22:35:38 on dist-test-slave-266d"
server_key: "f5b0b542c1d3a8004afe67a39dc7a98f"
server_key_iv: "67babba5bc0cd403c499d95b48295f6a"
server_key_version: "encryptionkey@0"
I20260808 22:35:38.621984 29553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:35:38.632301 29553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:35:38.634797 29553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:35:38.634953 29553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:35:38.641466 29553 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.190:42029
I20260808 22:35:38.641513 29670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.190:42029 every 8 connection(s)
I20260808 22:35:38.642333 29553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:35:38.644636 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 29553
I20260808 22:35:38.644768 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/master-0/wal/instance
I20260808 22:35:38.644892 29672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:35:38.645052 32094 external_mini_cluster.cc:1428] Setting key df9a9f68ebf9822a60d44d89b7ed83a5
I20260808 22:35:38.651716 29672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 589100aff40a4ecbb12e02db3aa96ab1: Bootstrap starting.
Aug 08 22:35:38 dist-test-slave-266d krb5kdc[29533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228538, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:35:38.654057 29672 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 589100aff40a4ecbb12e02db3aa96ab1: Neither blocks nor log segments found. Creating new log.
I20260808 22:35:38.654814 29672 log.cc:824] T 00000000000000000000000000000000 P 589100aff40a4ecbb12e02db3aa96ab1: Log is configured to *not* fsync() on all Append() calls
I20260808 22:35:38.656880 29672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 589100aff40a4ecbb12e02db3aa96ab1: No bootstrap required, opened a new log
I20260808 22:35:38.659051 29679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:38.646364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42590 (local address 127.31.87.190:42029)
0808 22:35:38.646783 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:38.646799 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:38.646841 (+ 42us) server_negotiation.cc:407] Connection header received
0808 22:35:38.647527 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:38.647549 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:38.647943 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:38.648307 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:38.648941 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:38.650402 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:38.651306 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:38.651956 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:38.652120 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:38.654620 (+ 2500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:38.654626 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:38.654641 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:35:38.654672 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:38.656354 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:35:38.656848 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:35:38.656852 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:35:38.656856 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:35:38.656956 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:35:38.657305 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:35:38.657307 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:35:38.657308 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:35:38.657679 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:38.657814 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:38.658124 (+ 310us) server_negotiation.cc:299] Negotiation successful
0808 22:35:38.658368 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":117,"threads_started":1}
I20260808 22:35:38.660817 29672 raft_consensus.cc:348] T 00000000000000000000000000000000 P 589100aff40a4ecbb12e02db3aa96ab1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "589100aff40a4ecbb12e02db3aa96ab1" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 42029 } }
I20260808 22:35:38.661129 29672 raft_consensus.cc:374] T 00000000000000000000000000000000 P 589100aff40a4ecbb12e02db3aa96ab1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:35:38.661201 29672 raft_consensus.cc:729] T 00000000000000000000000000000000 P 589100aff40a4ecbb12e02db3aa96ab1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 589100aff40a4ecbb12e02db3aa96ab1, State: Initialized, Role: FOLLOWER
I20260808 22:35:38.661634 29672 consensus_queue.cc:260] T 00000000000000000000000000000000 P 589100aff40a4ecbb12e02db3aa96ab1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "589100aff40a4ecbb12e02db3aa96ab1" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 42029 } }
I20260808 22:35:38.661767 29672 raft_consensus.cc:388] T 00000000000000000000000000000000 P 589100aff40a4ecbb12e02db3aa96ab1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:35:38.661834 29672 raft_consensus.cc:482] T 00000000000000000000000000000000 P 589100aff40a4ecbb12e02db3aa96ab1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:35:38.661933 29672 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 589100aff40a4ecbb12e02db3aa96ab1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:35:38.662777 29672 raft_consensus.cc:504] T 00000000000000000000000000000000 P 589100aff40a4ecbb12e02db3aa96ab1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "589100aff40a4ecbb12e02db3aa96ab1" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 42029 } }
I20260808 22:35:38.663106 29672 leader_election.cc:302] T 00000000000000000000000000000000 P 589100aff40a4ecbb12e02db3aa96ab1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 589100aff40a4ecbb12e02db3aa96ab1; no voters:
I20260808 22:35:38.663395 29672 leader_election.cc:288] T 00000000000000000000000000000000 P 589100aff40a4ecbb12e02db3aa96ab1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:35:38.663535 29681 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 589100aff40a4ecbb12e02db3aa96ab1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:35:38.663743 29681 raft_consensus.cc:686] T 00000000000000000000000000000000 P 589100aff40a4ecbb12e02db3aa96ab1 [term 1 LEADER]: Becoming Leader. State: Replica: 589100aff40a4ecbb12e02db3aa96ab1, State: Running, Role: LEADER
I20260808 22:35:38.664098 29681 consensus_queue.cc:237] T 00000000000000000000000000000000 P 589100aff40a4ecbb12e02db3aa96ab1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "589100aff40a4ecbb12e02db3aa96ab1" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 42029 } }
I20260808 22:35:38.664625 29672 sys_catalog.cc:564] T 00000000000000000000000000000000 P 589100aff40a4ecbb12e02db3aa96ab1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:35:38.665722 29683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 589100aff40a4ecbb12e02db3aa96ab1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 589100aff40a4ecbb12e02db3aa96ab1. Latest consensus state: current_term: 1 leader_uuid: "589100aff40a4ecbb12e02db3aa96ab1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "589100aff40a4ecbb12e02db3aa96ab1" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 42029 } } }
I20260808 22:35:38.665859 29683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 589100aff40a4ecbb12e02db3aa96ab1 [sys.catalog]: This master's current role is: LEADER
I20260808 22:35:38.666204 29682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 589100aff40a4ecbb12e02db3aa96ab1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "589100aff40a4ecbb12e02db3aa96ab1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "589100aff40a4ecbb12e02db3aa96ab1" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 42029 } } }
I20260808 22:35:38.666297 29694 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:35:38.667709 29682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 589100aff40a4ecbb12e02db3aa96ab1 [sys.catalog]: This master's current role is: LEADER
I20260808 22:35:38.669272 29694 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:35:38.674760 29694 catalog_manager.cc:1347] Generated new cluster ID: 5f6ba5bd5be342a5b90c36f046f57dff
I20260808 22:35:38.674810 29694 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:35:38.685375 29694 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:35:38.686204 29694 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:35:38.696851 29694 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 589100aff40a4ecbb12e02db3aa96ab1: Generated new TSK 0
I20260808 22:35:38.697453 29694 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.87.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab.
Entry for principal kudu/127.31.87.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab.
I20260808 22:35:38.731689 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.129:0
--local_ip_for_outbound_sockets=127.31.87.129
--webserver_interface=127.31.87.129
--webserver_port=0
--tserver_master_addrs=127.31.87.190:42029
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab
--principal=kudu/127.31.87.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:44005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:35:38.837813 29712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:35:38.838091 29712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:35:38.838155 29712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:35:38.842110 29712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:35:38.842161 29712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:35:38.842238 29712 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.129
I20260808 22:35:38.847254 29712 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:44005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.129.keytab
--principal=kudu/127.31.87.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.87.129
--webserver_port=0
--tserver_master_addrs=127.31.87.190:42029
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29712
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.129
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:35:38.848194 29712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:35:38.849179 29712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:35:38.858896 29731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:38.858912 29726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:38.859025 29729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:38.859007 29712 server_base.cc:1034] running on GCE node
I20260808 22:35:38.859526 29712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:35:38.860128 29712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:35:38.861279 29712 hybrid_clock.cc:648] HybridClock initialized: now 1786228538861260 us; error 31 us; skew 500 ppm
Aug 08 22:35:38 dist-test-slave-266d krb5kdc[29533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228538, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:35:38.863768 29712 init.cc:377] Logged in from keytab as kudu/127.31.87.129@KRBTEST.COM (short username kudu)
I20260808 22:35:38.864573 29712 webserver.cc:460] Webserver started at http://127.31.87.129:34313/ using document root <none> and password file <none>
I20260808 22:35:38.865172 29712 fs_manager.cc:359] Metadata directory not provided
I20260808 22:35:38.865244 29712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:35:38.865446 29712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:35:38.867468 29712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/ts-0/data/instance:
uuid: "3ba96d037bb24531b51e36ea04e355c8"
format_stamp: "Formatted at 2026-08-08 22:35:38 on dist-test-slave-266d"
server_key: "3d31b952b235a82c02c4c2081b05759c"
server_key_iv: "a875e71d86da2c6c33617c481f59bee4"
server_key_version: "encryptionkey@0"
I20260808 22:35:38.867995 29712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/ts-0/wal/instance:
uuid: "3ba96d037bb24531b51e36ea04e355c8"
format_stamp: "Formatted at 2026-08-08 22:35:38 on dist-test-slave-266d"
server_key: "3d31b952b235a82c02c4c2081b05759c"
server_key_iv: "a875e71d86da2c6c33617c481f59bee4"
server_key_version: "encryptionkey@0"
I20260808 22:35:38.871487 29712 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260808 22:35:38.873760 29742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:38.874897 29712 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 22:35:38.874994 29712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/ts-0/wal
uuid: "3ba96d037bb24531b51e36ea04e355c8"
format_stamp: "Formatted at 2026-08-08 22:35:38 on dist-test-slave-266d"
server_key: "3d31b952b235a82c02c4c2081b05759c"
server_key_iv: "a875e71d86da2c6c33617c481f59bee4"
server_key_version: "encryptionkey@0"
I20260808 22:35:38.875090 29712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:35:38.887660 29712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:35:38.890261 29712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:35:38.890420 29712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:35:38.890949 29712 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:35:38.891713 29712 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:35:38.891758 29712 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:38.891798 29712 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:35:38.891826 29712 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:38.899835 29712 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.129:32959
I20260808 22:35:38.899889 29926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.129:32959 every 8 connection(s)
I20260808 22:35:38.900640 29712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
I20260808 22:35:38.907130 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 29712
I20260808 22:35:38.907256 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/ts-0/wal/instance
I20260808 22:35:38.907568 32094 external_mini_cluster.cc:1428] Setting key 171b9378981f820628eee822312f5fb6
Aug 08 22:35:38 dist-test-slave-266d krb5kdc[29533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228538, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.129@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:35:38.914028 29679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:38.902311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:50087 (local address 127.31.87.190:42029)
0808 22:35:38.902447 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:38.902451 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:38.903208 (+ 757us) server_negotiation.cc:407] Connection header received
0808 22:35:38.904484 (+ 1276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:38.904488 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:38.904545 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:38.904628 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:38.905786 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:38.906322 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:38.907705 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:38.907964 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:38.908080 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:38.910680 (+ 2600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:38.910683 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:38.910686 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:35:38.910711 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:38.912123 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:35:38.912568 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:35:38.912571 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:35:38.912572 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:35:38.912658 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:35:38.913005 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:35:38.913006 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:35:38.913007 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:35:38.913210 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:38.913289 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:38.913813 (+ 524us) server_negotiation.cc:299] Negotiation successful
0808 22:35:38.913909 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 22:35:38.915110 29930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:38.902653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:42029 (local address 127.31.87.129:50087)
0808 22:35:38.903062 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:38.903112 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 22:35:38.904278 (+ 1166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:35:38.904799 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:35:38.904806 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:35:38.905174 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:35:38.905645 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:38.905659 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:38.906443 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:38.906446 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:38.907581 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:38.907594 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:38.908127 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:38.908131 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:38.908328 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:38.908878 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:35:38.908899 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:35:38.910515 (+ 1616us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:35:38.912232 (+ 1717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:35:38.912236 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:35:38.912247 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:35:38.912485 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:35:38.912755 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:35:38.912758 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:35:38.912759 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:35:38.912925 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:35:38.913306 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:35:38.913312 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:35:38.913588 (+ 276us) client_negotiation.cc:769] Sending connection context
0808 22:35:38.913790 (+ 202us) client_negotiation.cc:240] Negotiation successful
0808 22:35:38.914133 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":115,"threads_started":1}
I20260808 22:35:38.916319 29927 heartbeater.cc:347] Connected to a master server at 127.31.87.190:42029
I20260808 22:35:38.916536 29927 heartbeater.cc:464] Registering TS with master...
I20260808 22:35:38.916986 29927 heartbeater.cc:511] Master 127.31.87.190:42029 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.31.87.130@KRBTEST.COM; defaulting to no policy
I20260808 22:35:38.918465 29612 ts_manager.cc:194] Registered new tserver with Master: 3ba96d037bb24531b51e36ea04e355c8 (127.31.87.129:32959)
I20260808 22:35:38.919600 29612 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.129@KRBTEST.COM'} at 127.31.87.129:50087
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab.
Entry for principal kudu/127.31.87.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab.
I20260808 22:35:38.935464 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.130:0
--local_ip_for_outbound_sockets=127.31.87.130
--webserver_interface=127.31.87.130
--webserver_port=0
--tserver_master_addrs=127.31.87.190:42029
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab
--principal=kudu/127.31.87.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:44005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:35:39.042657 29935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:35:39.042894 29935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:35:39.042937 29935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:35:39.046846 29935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:35:39.046903 29935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:35:39.046974 29935 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.130
I20260808 22:35:39.052043 29935 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:44005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.130.keytab
--principal=kudu/127.31.87.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.87.130
--webserver_port=0
--tserver_master_addrs=127.31.87.190:42029
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29935
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.130
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:35:39.052978 29935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:35:39.053929 29935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:35:39.063184 29944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:39.063206 29943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:39.063266 29935 server_base.cc:1034] running on GCE node
W20260808 22:35:39.063217 29947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:39.063755 29935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:35:39.064364 29935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:35:39.065529 29935 hybrid_clock.cc:648] HybridClock initialized: now 1786228539065505 us; error 34 us; skew 500 ppm
Aug 08 22:35:39 dist-test-slave-266d krb5kdc[29533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228539, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:35:39.068056 29935 init.cc:377] Logged in from keytab as kudu/127.31.87.130@KRBTEST.COM (short username kudu)
I20260808 22:35:39.068877 29935 webserver.cc:460] Webserver started at http://127.31.87.130:42649/ using document root <none> and password file <none>
I20260808 22:35:39.069433 29935 fs_manager.cc:359] Metadata directory not provided
I20260808 22:35:39.069502 29935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:35:39.069715 29935 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:35:39.071648 29935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/ts-1/data/instance:
uuid: "7746540492264da6aa1463613c1eff03"
format_stamp: "Formatted at 2026-08-08 22:35:39 on dist-test-slave-266d"
server_key: "3660fcab22195190f0c3629dc78d4737"
server_key_iv: "e9f4dd1cee7c7356af94b57263782806"
server_key_version: "encryptionkey@0"
I20260808 22:35:39.072170 29935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/ts-1/wal/instance:
uuid: "7746540492264da6aa1463613c1eff03"
format_stamp: "Formatted at 2026-08-08 22:35:39 on dist-test-slave-266d"
server_key: "3660fcab22195190f0c3629dc78d4737"
server_key_iv: "e9f4dd1cee7c7356af94b57263782806"
server_key_version: "encryptionkey@0"
I20260808 22:35:39.075636 29935 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260808 22:35:39.077924 29960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:39.079155 29935 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260808 22:35:39.079246 29935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/ts-1/wal
uuid: "7746540492264da6aa1463613c1eff03"
format_stamp: "Formatted at 2026-08-08 22:35:39 on dist-test-slave-266d"
server_key: "3660fcab22195190f0c3629dc78d4737"
server_key_iv: "e9f4dd1cee7c7356af94b57263782806"
server_key_version: "encryptionkey@0"
I20260808 22:35:39.079324 29935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:35:39.096354 29935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:35:39.099182 29935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:35:39.099318 29935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:35:39.099790 29935 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:35:39.100495 29935 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:35:39.100539 29935 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:39.100575 29935 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:35:39.100595 29935 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:39.108723 29935 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.130:33607
I20260808 22:35:39.108752 30149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.130:33607 every 8 connection(s)
I20260808 22:35:39.109530 29935 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
I20260808 22:35:39.111227 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 29935
I20260808 22:35:39.111371 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/ts-1/wal/instance
I20260808 22:35:39.111683 32094 external_mini_cluster.cc:1428] Setting key 1c4ad68108337bbadae948b7eda76d1d
Aug 08 22:35:39 dist-test-slave-266d krb5kdc[29533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228539, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.130@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.87.131@KRBTEST.COM; defaulting to no policy
I20260808 22:35:39.122599 29679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:39.111461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:34375 (local address 127.31.87.190:42029)
0808 22:35:39.111572 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:39.111576 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:39.112175 (+ 599us) server_negotiation.cc:407] Connection header received
0808 22:35:39.113569 (+ 1394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:39.113572 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:39.113637 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:39.113709 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:39.114765 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:39.115281 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:39.116373 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:39.116598 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:39.116694 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:39.119148 (+ 2454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:39.119153 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:39.119156 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:35:39.119184 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:39.120716 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:35:39.121208 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:35:39.121212 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:35:39.121214 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:35:39.121304 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:35:39.121600 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:35:39.121603 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:35:39.121605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:35:39.121839 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:39.121930 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:39.122340 (+ 410us) server_negotiation.cc:299] Negotiation successful
0808 22:35:39.122436 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 22:35:39.123286 30152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:39.111548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:42029 (local address 127.31.87.130:34375)
0808 22:35:39.112009 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:39.112054 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 22:35:39.113382 (+ 1328us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:35:39.113879 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:35:39.113885 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:35:39.114201 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:35:39.114633 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:39.114646 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:39.115397 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:39.115401 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:39.116254 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:39.116265 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:39.116741 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:39.116745 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:39.116902 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:39.117451 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:35:39.117473 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:35:39.118986 (+ 1513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:35:39.120839 (+ 1853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:35:39.120843 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:35:39.120854 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:35:39.121083 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:35:39.121385 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:35:39.121387 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:35:39.121388 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:35:39.121506 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:35:39.121932 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:35:39.121937 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:35:39.122146 (+ 209us) client_negotiation.cc:769] Sending connection context
0808 22:35:39.122287 (+ 141us) client_negotiation.cc:240] Negotiation successful
0808 22:35:39.122509 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":148,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.131@KRBTEST.COM" created.
I20260808 22:35:39.124362 30150 heartbeater.cc:347] Connected to a master server at 127.31.87.190:42029
I20260808 22:35:39.124586 30150 heartbeater.cc:464] Registering TS with master...
I20260808 22:35:39.125157 30150 heartbeater.cc:511] Master 127.31.87.190:42029 requested a full tablet report, sending...
I20260808 22:35:39.126266 29612 ts_manager.cc:194] Registered new tserver with Master: 7746540492264da6aa1463613c1eff03 (127.31.87.130:33607)
I20260808 22:35:39.127032 29612 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.130@KRBTEST.COM'} at 127.31.87.130:34375
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab.
Entry for principal kudu/127.31.87.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab.
I20260808 22:35:39.137931 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.131:0
--local_ip_for_outbound_sockets=127.31.87.131
--webserver_interface=127.31.87.131
--webserver_port=0
--tserver_master_addrs=127.31.87.190:42029
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab
--principal=kudu/127.31.87.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:44005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:35:39.243468 30157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:35:39.243701 30157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:35:39.243745 30157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:35:39.247613 30157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:35:39.247661 30157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:35:39.247728 30157 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.131
I20260808 22:35:39.252684 30157 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:44005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.131.keytab
--principal=kudu/127.31.87.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.87.131
--webserver_port=0
--tserver_master_addrs=127.31.87.190:42029
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30157
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.131
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:35:39.253584 30157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:35:39.254477 30157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:35:39.263650 30166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:39.263710 30163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:39.263710 30164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:39.263885 30157 server_base.cc:1034] running on GCE node
I20260808 22:35:39.264338 30157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:35:39.264915 30157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:35:39.266089 30157 hybrid_clock.cc:648] HybridClock initialized: now 1786228539266080 us; error 53 us; skew 500 ppm
Aug 08 22:35:39 dist-test-slave-266d krb5kdc[29533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228539, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:35:39.271334 30157 init.cc:377] Logged in from keytab as kudu/127.31.87.131@KRBTEST.COM (short username kudu)
I20260808 22:35:39.272145 30157 webserver.cc:460] Webserver started at http://127.31.87.131:32865/ using document root <none> and password file <none>
I20260808 22:35:39.272634 30157 fs_manager.cc:359] Metadata directory not provided
I20260808 22:35:39.272691 30157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:35:39.272853 30157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:35:39.274772 30157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/ts-2/data/instance:
uuid: "13991b98219742f6ba4b13205a451857"
format_stamp: "Formatted at 2026-08-08 22:35:39 on dist-test-slave-266d"
server_key: "40a27510fbc335d6b7eda22c9320baea"
server_key_iv: "4800218c578a0e195f6e8365b6870bb3"
server_key_version: "encryptionkey@0"
I20260808 22:35:39.275261 30157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/ts-2/wal/instance:
uuid: "13991b98219742f6ba4b13205a451857"
format_stamp: "Formatted at 2026-08-08 22:35:39 on dist-test-slave-266d"
server_key: "40a27510fbc335d6b7eda22c9320baea"
server_key_iv: "4800218c578a0e195f6e8365b6870bb3"
server_key_version: "encryptionkey@0"
I20260808 22:35:39.279304 30157 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 22:35:39.282254 30174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:39.283648 30157 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 22:35:39.283763 30157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/ts-2/wal
uuid: "13991b98219742f6ba4b13205a451857"
format_stamp: "Formatted at 2026-08-08 22:35:39 on dist-test-slave-266d"
server_key: "40a27510fbc335d6b7eda22c9320baea"
server_key_iv: "4800218c578a0e195f6e8365b6870bb3"
server_key_version: "encryptionkey@0"
I20260808 22:35:39.283867 30157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:35:39.312515 30157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:35:39.317437 30157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:35:39.318696 30157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:35:39.319582 30157 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:35:39.320667 30157 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:35:39.320731 30157 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:39.320787 30157 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:35:39.320818 30157 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:39.334087 30157 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.131:40525
I20260808 22:35:39.334136 30349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.131:40525 every 8 connection(s)
I20260808 22:35:39.335177 30157 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
I20260808 22:35:39.343559 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 30157
I20260808 22:35:39.343680 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/ts-2/wal/instance
I20260808 22:35:39.343945 32094 external_mini_cluster.cc:1428] Setting key 6a885f3ad1e91ffc9dc78806b90a90c0
Aug 08 22:35:39 dist-test-slave-266d krb5kdc[29533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228539, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.131@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:35:39.351858 29679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:39.338742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:54573 (local address 127.31.87.190:42029)
0808 22:35:39.338885 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:39.338890 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:39.338907 (+ 17us) server_negotiation.cc:407] Connection header received
0808 22:35:39.340018 (+ 1111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:39.340022 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:39.340075 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:39.340167 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:39.341716 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:39.342541 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:39.344202 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:39.344498 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:39.344570 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:39.347972 (+ 3402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:39.347975 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:39.347978 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:35:39.348004 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:39.349619 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:35:39.350190 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:35:39.350194 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:35:39.350196 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:35:39.350279 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:35:39.350623 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:35:39.350626 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:35:39.350628 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:35:39.350830 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:39.350902 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:39.351557 (+ 655us) server_negotiation.cc:299] Negotiation successful
0808 22:35:39.351692 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 22:35:39.353360 30352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:39.337675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:42029 (local address 127.31.87.131:54573)
0808 22:35:39.338195 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:39.338250 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 22:35:39.339785 (+ 1535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:35:39.340459 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:35:39.340468 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:35:39.340906 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:35:39.341548 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:39.341565 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:39.342701 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:39.342704 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:39.344041 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:39.344054 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:39.344772 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:39.344776 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:39.344961 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:39.346009 (+ 1048us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:35:39.346028 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:35:39.347795 (+ 1767us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:35:39.349744 (+ 1949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:35:39.349750 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:35:39.349765 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:35:39.350069 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:35:39.350374 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:35:39.350377 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:35:39.350379 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:35:39.350529 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:35:39.350963 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:35:39.350968 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:35:39.351263 (+ 295us) client_negotiation.cc:769] Sending connection context
0808 22:35:39.352014 (+ 751us) client_negotiation.cc:240] Negotiation successful
0808 22:35:39.352313 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":131,"threads_started":1}
I20260808 22:35:39.354518 30350 heartbeater.cc:347] Connected to a master server at 127.31.87.190:42029
I20260808 22:35:39.354771 30350 heartbeater.cc:464] Registering TS with master...
I20260808 22:35:39.355319 30350 heartbeater.cc:511] Master 127.31.87.190:42029 requested a full tablet report, sending...
I20260808 22:35:39.356431 29612 ts_manager.cc:194] Registered new tserver with Master: 13991b98219742f6ba4b13205a451857 (127.31.87.131:40525)
I20260808 22:35:39.357142 29612 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.87.131@KRBTEST.COM'} at 127.31.87.131:54573
I20260808 22:35:39.361872 32094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:35:39.371493 29679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:39.364080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42600 (local address 127.31.87.190:42029)
0808 22:35:39.364194 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:39.364199 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:39.364382 (+ 183us) server_negotiation.cc:407] Connection header received
0808 22:35:39.364498 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:39.364500 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:39.364536 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:39.364596 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:39.365490 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:39.365964 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:39.367120 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:39.367327 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:39.367399 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:39.368841 (+ 1442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:39.368844 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:39.368846 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:35:39.368868 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:39.370077 (+ 1209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:35:39.370508 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:35:39.370511 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:35:39.370512 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:35:39.370573 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:35:39.370803 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:35:39.370804 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:35:39.370805 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:35:39.370979 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:39.371074 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:39.371295 (+ 221us) server_negotiation.cc:299] Negotiation successful
0808 22:35:39.371381 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 22:35:39.374086 29612 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42600:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:35:39.376935 29612 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:35:39.391780 30373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:39.386347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.131:40525 (local address 127.0.0.1:35314)
0808 22:35:39.386588 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:39.386600 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:35:39.386681 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:35:39.387668 (+ 987us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:35:39.387671 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:35:39.387690 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:35:39.387785 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:39.387810 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:39.388805 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:39.388809 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:39.390363 (+ 1554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:39.390376 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:39.391195 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:39.391199 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:39.391362 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:39.391408 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 22:35:39.391519 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 22:35:39.391637 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":182,"thread_start_us":105,"threads_started":1}
I20260808 22:35:39.391788 30374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:39.386784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35314 (local address 127.31.87.131:40525)
0808 22:35:39.387134 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:39.387145 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:39.387169 (+ 24us) server_negotiation.cc:407] Connection header received
0808 22:35:39.387244 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:39.387248 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:39.387455 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:39.387684 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:35:39.387905 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:39.388678 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:39.390480 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:39.391075 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:39.391174 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:39.391325 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:39.391549 (+ 224us) server_negotiation.cc:299] Negotiation successful
0808 22:35:39.391651 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":171,"threads_started":1}
I20260808 22:35:39.392426 30368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:39.385628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:32959 (local address 127.0.0.1:45824)
0808 22:35:39.386151 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:39.386200 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 22:35:39.386326 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:35:39.387049 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:35:39.387055 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:35:39.387078 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:35:39.387207 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:39.387215 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:39.388854 (+ 1639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:39.388857 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:39.390156 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:39.390167 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:39.391275 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:39.391279 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:39.392138 (+ 859us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:39.392163 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 22:35:39.392237 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 22:35:39.392300 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"mutex_wait_us":775,"thread_start_us":95,"threads_started":1}
I20260808 22:35:39.392566 30371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:39.385902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45824 (local address 127.31.87.129:32959)
0808 22:35:39.386271 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:39.386285 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:39.386311 (+ 26us) server_negotiation.cc:407] Connection header received
0808 22:35:39.386710 (+ 399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:39.386715 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:39.386873 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:39.387363 (+ 490us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:35:39.387511 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:39.388696 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:39.390311 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:39.391140 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:39.391356 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:39.391497 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:39.392286 (+ 789us) server_negotiation.cc:299] Negotiation successful
0808 22:35:39.392426 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":110,"threads_started":1}
I20260808 22:35:39.392884 30364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:39.385414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41318 (local address 127.31.87.130:33607)
0808 22:35:39.385753 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:39.385765 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:39.386535 (+ 770us) server_negotiation.cc:407] Connection header received
0808 22:35:39.386662 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:39.386668 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:39.386873 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:39.387049 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:35:39.387664 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:39.388763 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:39.391462 (+ 2699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:39.392096 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:39.392186 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:39.392310 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:39.392656 (+ 346us) server_negotiation.cc:299] Negotiation successful
0808 22:35:39.392757 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":150,"threads_started":1}
I20260808 22:35:39.393298 30366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:39.385807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.130:33607 (local address 127.0.0.1:41318)
0808 22:35:39.386440 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:39.386453 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:35:39.386548 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:35:39.387424 (+ 876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:35:39.387428 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:35:39.387450 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:35:39.387531 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:39.387537 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:39.388900 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:39.388903 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:39.391336 (+ 2433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:39.391345 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:39.392411 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:39.392414 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:39.392498 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:39.392518 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 22:35:39.393071 (+ 553us) client_negotiation.cc:240] Negotiation successful
0808 22:35:39.393177 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"mutex_wait_us":1260,"thread_start_us":108,"threads_started":1}
I20260808 22:35:39.394867 30257 tablet_service.cc:1467] Processing CreateTablet for tablet bacf103945724ed894ed6751ce3ed61c (DEFAULT_TABLE table=test-table [id=151d7d088cb14d4fb9f3d6d00c0512ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:35:39.395799 30257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bacf103945724ed894ed6751ce3ed61c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:35:39.395428 29814 tablet_service.cc:1467] Processing CreateTablet for tablet bacf103945724ed894ed6751ce3ed61c (DEFAULT_TABLE table=test-table [id=151d7d088cb14d4fb9f3d6d00c0512ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:35:39.396453 29814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bacf103945724ed894ed6751ce3ed61c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:35:39.396159 30041 tablet_service.cc:1467] Processing CreateTablet for tablet bacf103945724ed894ed6751ce3ed61c (DEFAULT_TABLE table=test-table [id=151d7d088cb14d4fb9f3d6d00c0512ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:35:39.397163 30041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bacf103945724ed894ed6751ce3ed61c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:35:39.403118 30375 tablet_bootstrap.cc:492] T bacf103945724ed894ed6751ce3ed61c P 13991b98219742f6ba4b13205a451857: Bootstrap starting.
I20260808 22:35:39.404644 30378 tablet_bootstrap.cc:492] T bacf103945724ed894ed6751ce3ed61c P 7746540492264da6aa1463613c1eff03: Bootstrap starting.
I20260808 22:35:39.405478 30375 tablet_bootstrap.cc:654] T bacf103945724ed894ed6751ce3ed61c P 13991b98219742f6ba4b13205a451857: Neither blocks nor log segments found. Creating new log.
I20260808 22:35:39.406448 30378 tablet_bootstrap.cc:654] T bacf103945724ed894ed6751ce3ed61c P 7746540492264da6aa1463613c1eff03: Neither blocks nor log segments found. Creating new log.
I20260808 22:35:39.406864 30375 log.cc:824] T bacf103945724ed894ed6751ce3ed61c P 13991b98219742f6ba4b13205a451857: Log is configured to *not* fsync() on all Append() calls
I20260808 22:35:39.407528 30378 log.cc:824] T bacf103945724ed894ed6751ce3ed61c P 7746540492264da6aa1463613c1eff03: Log is configured to *not* fsync() on all Append() calls
I20260808 22:35:39.408197 30377 tablet_bootstrap.cc:492] T bacf103945724ed894ed6751ce3ed61c P 3ba96d037bb24531b51e36ea04e355c8: Bootstrap starting.
I20260808 22:35:39.408900 30375 tablet_bootstrap.cc:492] T bacf103945724ed894ed6751ce3ed61c P 13991b98219742f6ba4b13205a451857: No bootstrap required, opened a new log
I20260808 22:35:39.409168 30375 ts_tablet_manager.cc:1397] T bacf103945724ed894ed6751ce3ed61c P 13991b98219742f6ba4b13205a451857: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260808 22:35:39.409862 30378 tablet_bootstrap.cc:492] T bacf103945724ed894ed6751ce3ed61c P 7746540492264da6aa1463613c1eff03: No bootstrap required, opened a new log
I20260808 22:35:39.410084 30378 ts_tablet_manager.cc:1397] T bacf103945724ed894ed6751ce3ed61c P 7746540492264da6aa1463613c1eff03: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 22:35:39.411095 30377 tablet_bootstrap.cc:654] T bacf103945724ed894ed6751ce3ed61c P 3ba96d037bb24531b51e36ea04e355c8: Neither blocks nor log segments found. Creating new log.
I20260808 22:35:39.412321 30377 log.cc:824] T bacf103945724ed894ed6751ce3ed61c P 3ba96d037bb24531b51e36ea04e355c8: Log is configured to *not* fsync() on all Append() calls
I20260808 22:35:39.413189 30375 raft_consensus.cc:348] T bacf103945724ed894ed6751ce3ed61c P 13991b98219742f6ba4b13205a451857 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7746540492264da6aa1463613c1eff03" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 33607 } } peers { permanent_uuid: "13991b98219742f6ba4b13205a451857" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 40525 } } peers { permanent_uuid: "3ba96d037bb24531b51e36ea04e355c8" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 32959 } }
I20260808 22:35:39.413456 30375 raft_consensus.cc:374] T bacf103945724ed894ed6751ce3ed61c P 13991b98219742f6ba4b13205a451857 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:35:39.413534 30375 raft_consensus.cc:729] T bacf103945724ed894ed6751ce3ed61c P 13991b98219742f6ba4b13205a451857 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13991b98219742f6ba4b13205a451857, State: Initialized, Role: FOLLOWER
I20260808 22:35:39.413501 30378 raft_consensus.cc:348] T bacf103945724ed894ed6751ce3ed61c P 7746540492264da6aa1463613c1eff03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7746540492264da6aa1463613c1eff03" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 33607 } } peers { permanent_uuid: "13991b98219742f6ba4b13205a451857" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 40525 } } peers { permanent_uuid: "3ba96d037bb24531b51e36ea04e355c8" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 32959 } }
I20260808 22:35:39.413730 30378 raft_consensus.cc:374] T bacf103945724ed894ed6751ce3ed61c P 7746540492264da6aa1463613c1eff03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:35:39.413798 30378 raft_consensus.cc:729] T bacf103945724ed894ed6751ce3ed61c P 7746540492264da6aa1463613c1eff03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7746540492264da6aa1463613c1eff03, State: Initialized, Role: FOLLOWER
I20260808 22:35:39.414027 30375 consensus_queue.cc:260] T bacf103945724ed894ed6751ce3ed61c P 13991b98219742f6ba4b13205a451857 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7746540492264da6aa1463613c1eff03" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 33607 } } peers { permanent_uuid: "13991b98219742f6ba4b13205a451857" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 40525 } } peers { permanent_uuid: "3ba96d037bb24531b51e36ea04e355c8" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 32959 } }
I20260808 22:35:39.414693 30350 heartbeater.cc:503] Master 127.31.87.190:42029 was elected leader, sending a full tablet report...
I20260808 22:35:39.414180 30378 consensus_queue.cc:260] T bacf103945724ed894ed6751ce3ed61c P 7746540492264da6aa1463613c1eff03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7746540492264da6aa1463613c1eff03" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 33607 } } peers { permanent_uuid: "13991b98219742f6ba4b13205a451857" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 40525 } } peers { permanent_uuid: "3ba96d037bb24531b51e36ea04e355c8" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 32959 } }
I20260808 22:35:39.415004 30375 ts_tablet_manager.cc:1428] T bacf103945724ed894ed6751ce3ed61c P 13991b98219742f6ba4b13205a451857: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 22:35:39.415416 30150 heartbeater.cc:503] Master 127.31.87.190:42029 was elected leader, sending a full tablet report...
I20260808 22:35:39.416939 30377 tablet_bootstrap.cc:492] T bacf103945724ed894ed6751ce3ed61c P 3ba96d037bb24531b51e36ea04e355c8: No bootstrap required, opened a new log
I20260808 22:35:39.417142 30377 ts_tablet_manager.cc:1397] T bacf103945724ed894ed6751ce3ed61c P 3ba96d037bb24531b51e36ea04e355c8: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260808 22:35:39.417799 30378 ts_tablet_manager.cc:1428] T bacf103945724ed894ed6751ce3ed61c P 7746540492264da6aa1463613c1eff03: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20260808 22:35:39.420969 30377 raft_consensus.cc:348] T bacf103945724ed894ed6751ce3ed61c P 3ba96d037bb24531b51e36ea04e355c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7746540492264da6aa1463613c1eff03" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 33607 } } peers { permanent_uuid: "13991b98219742f6ba4b13205a451857" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 40525 } } peers { permanent_uuid: "3ba96d037bb24531b51e36ea04e355c8" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 32959 } }
I20260808 22:35:39.421367 30377 raft_consensus.cc:374] T bacf103945724ed894ed6751ce3ed61c P 3ba96d037bb24531b51e36ea04e355c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:35:39.421455 30377 raft_consensus.cc:729] T bacf103945724ed894ed6751ce3ed61c P 3ba96d037bb24531b51e36ea04e355c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ba96d037bb24531b51e36ea04e355c8, State: Initialized, Role: FOLLOWER
I20260808 22:35:39.421977 30377 consensus_queue.cc:260] T bacf103945724ed894ed6751ce3ed61c P 3ba96d037bb24531b51e36ea04e355c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7746540492264da6aa1463613c1eff03" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 33607 } } peers { permanent_uuid: "13991b98219742f6ba4b13205a451857" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 40525 } } peers { permanent_uuid: "3ba96d037bb24531b51e36ea04e355c8" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 32959 } }
I20260808 22:35:39.422544 29927 heartbeater.cc:503] Master 127.31.87.190:42029 was elected leader, sending a full tablet report...
I20260808 22:35:39.422971 30377 ts_tablet_manager.cc:1428] T bacf103945724ed894ed6751ce3ed61c P 3ba96d037bb24531b51e36ea04e355c8: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 22:35:39.429348 30385 raft_consensus.cc:482] T bacf103945724ed894ed6751ce3ed61c P 7746540492264da6aa1463613c1eff03 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:35:39.429478 30385 raft_consensus.cc:504] T bacf103945724ed894ed6751ce3ed61c P 7746540492264da6aa1463613c1eff03 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7746540492264da6aa1463613c1eff03" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 33607 } } peers { permanent_uuid: "13991b98219742f6ba4b13205a451857" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 40525 } } peers { permanent_uuid: "3ba96d037bb24531b51e36ea04e355c8" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 32959 } }
I20260808 22:35:39.430387 30385 leader_election.cc:288] T bacf103945724ed894ed6751ce3ed61c P 7746540492264da6aa1463613c1eff03 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13991b98219742f6ba4b13205a451857 (127.31.87.131:40525), 3ba96d037bb24531b51e36ea04e355c8 (127.31.87.129:32959)
I20260808 22:35:39.434836 30371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:39.430698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:50351 (local address 127.31.87.129:32959)
0808 22:35:39.430805 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:39.430809 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:39.430822 (+ 13us) server_negotiation.cc:407] Connection header received
0808 22:35:39.430933 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:39.430935 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:39.430971 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:39.431047 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:35:39.431302 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:39.432052 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:39.433639 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:39.434242 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:39.434359 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:39.434409 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:39.434650 (+ 241us) server_negotiation.cc:299] Negotiation successful
0808 22:35:39.434719 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 22:35:39.434836 30374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:39.430674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:51815 (local address 127.31.87.131:40525)
0808 22:35:39.430787 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:39.430791 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:39.431017 (+ 226us) server_negotiation.cc:407] Connection header received
0808 22:35:39.431100 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:39.431104 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:39.431147 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:39.431207 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:35:39.431436 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:39.432135 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:39.433650 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:39.434242 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:39.434359 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:39.434409 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:39.434668 (+ 259us) server_negotiation.cc:299] Negotiation successful
0808 22:35:39.434725 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 22:35:39.434865 30152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:39.430580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:32959 (local address 127.31.87.130:50351)
0808 22:35:39.430752 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:39.430766 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:35:39.430845 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:35:39.431061 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:35:39.431064 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:35:39.431078 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:35:39.431198 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:39.431204 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:39.432155 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:39.432158 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:39.433521 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:39.433532 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:39.434350 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:39.434354 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:39.434512 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:39.434559 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 22:35:39.434636 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 22:35:39.434722 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"mutex_wait_us":109}
I20260808 22:35:39.434878 30388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:39.430622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.131:40525 (local address 127.31.87.130:51815)
0808 22:35:39.430937 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:39.430948 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:35:39.431027 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:35:39.431227 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:35:39.431229 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:35:39.431244 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:35:39.431353 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:39.431357 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:39.432217 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:39.432220 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:39.433521 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:39.433532 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:39.434350 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:39.434354 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:39.434528 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:39.434565 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 22:35:39.434654 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 22:35:39.434728 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":99,"thread_start_us":95,"threads_started":1}
I20260808 22:35:39.435396 29844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bacf103945724ed894ed6751ce3ed61c" candidate_uuid: "7746540492264da6aa1463613c1eff03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ba96d037bb24531b51e36ea04e355c8" is_pre_election: true
I20260808 22:35:39.435581 30282 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bacf103945724ed894ed6751ce3ed61c" candidate_uuid: "7746540492264da6aa1463613c1eff03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13991b98219742f6ba4b13205a451857" is_pre_election: true
I20260808 22:35:39.435649 29844 raft_consensus.cc:2454] T bacf103945724ed894ed6751ce3ed61c P 3ba96d037bb24531b51e36ea04e355c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7746540492264da6aa1463613c1eff03 in term 0.
I20260808 22:35:39.435840 30282 raft_consensus.cc:2454] T bacf103945724ed894ed6751ce3ed61c P 13991b98219742f6ba4b13205a451857 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7746540492264da6aa1463613c1eff03 in term 0.
I20260808 22:35:39.436095 29962 leader_election.cc:302] T bacf103945724ed894ed6751ce3ed61c P 7746540492264da6aa1463613c1eff03 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ba96d037bb24531b51e36ea04e355c8, 7746540492264da6aa1463613c1eff03; no voters:
I20260808 22:35:39.436298 30385 raft_consensus.cc:2781] T bacf103945724ed894ed6751ce3ed61c P 7746540492264da6aa1463613c1eff03 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:35:39.436388 30385 raft_consensus.cc:482] T bacf103945724ed894ed6751ce3ed61c P 7746540492264da6aa1463613c1eff03 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:35:39.436439 30385 raft_consensus.cc:3037] T bacf103945724ed894ed6751ce3ed61c P 7746540492264da6aa1463613c1eff03 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:35:39.437477 30385 raft_consensus.cc:504] T bacf103945724ed894ed6751ce3ed61c P 7746540492264da6aa1463613c1eff03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7746540492264da6aa1463613c1eff03" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 33607 } } peers { permanent_uuid: "13991b98219742f6ba4b13205a451857" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 40525 } } peers { permanent_uuid: "3ba96d037bb24531b51e36ea04e355c8" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 32959 } }
I20260808 22:35:39.437800 30385 leader_election.cc:288] T bacf103945724ed894ed6751ce3ed61c P 7746540492264da6aa1463613c1eff03 [CANDIDATE]: Term 1 election: Requested vote from peers 13991b98219742f6ba4b13205a451857 (127.31.87.131:40525), 3ba96d037bb24531b51e36ea04e355c8 (127.31.87.129:32959)
I20260808 22:35:39.438078 30282 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bacf103945724ed894ed6751ce3ed61c" candidate_uuid: "7746540492264da6aa1463613c1eff03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13991b98219742f6ba4b13205a451857"
I20260808 22:35:39.438189 30282 raft_consensus.cc:3037] T bacf103945724ed894ed6751ce3ed61c P 13991b98219742f6ba4b13205a451857 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:35:39.438284 29844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bacf103945724ed894ed6751ce3ed61c" candidate_uuid: "7746540492264da6aa1463613c1eff03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ba96d037bb24531b51e36ea04e355c8"
I20260808 22:35:39.438393 29844 raft_consensus.cc:3037] T bacf103945724ed894ed6751ce3ed61c P 3ba96d037bb24531b51e36ea04e355c8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:35:39.439191 30282 raft_consensus.cc:2454] T bacf103945724ed894ed6751ce3ed61c P 13991b98219742f6ba4b13205a451857 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7746540492264da6aa1463613c1eff03 in term 1.
I20260808 22:35:39.439361 29844 raft_consensus.cc:2454] T bacf103945724ed894ed6751ce3ed61c P 3ba96d037bb24531b51e36ea04e355c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7746540492264da6aa1463613c1eff03 in term 1.
I20260808 22:35:39.439462 29961 leader_election.cc:302] T bacf103945724ed894ed6751ce3ed61c P 7746540492264da6aa1463613c1eff03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13991b98219742f6ba4b13205a451857, 7746540492264da6aa1463613c1eff03; no voters:
I20260808 22:35:39.439642 30385 raft_consensus.cc:2781] T bacf103945724ed894ed6751ce3ed61c P 7746540492264da6aa1463613c1eff03 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:35:39.439899 30385 raft_consensus.cc:686] T bacf103945724ed894ed6751ce3ed61c P 7746540492264da6aa1463613c1eff03 [term 1 LEADER]: Becoming Leader. State: Replica: 7746540492264da6aa1463613c1eff03, State: Running, Role: LEADER
I20260808 22:35:39.440186 30385 consensus_queue.cc:237] T bacf103945724ed894ed6751ce3ed61c P 7746540492264da6aa1463613c1eff03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7746540492264da6aa1463613c1eff03" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 33607 } } peers { permanent_uuid: "13991b98219742f6ba4b13205a451857" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 40525 } } peers { permanent_uuid: "3ba96d037bb24531b51e36ea04e355c8" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 32959 } }
I20260808 22:35:39.443112 29611 catalog_manager.cc:5495] T bacf103945724ed894ed6751ce3ed61c P 7746540492264da6aa1463613c1eff03 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7746540492264da6aa1463613c1eff03 (127.31.87.130). New cstate: current_term: 1 leader_uuid: "7746540492264da6aa1463613c1eff03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7746540492264da6aa1463613c1eff03" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 33607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13991b98219742f6ba4b13205a451857" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 40525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ba96d037bb24531b51e36ea04e355c8" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 32959 } health_report { overall_health: UNKNOWN } } }
I20260808 22:35:39.459410 30364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:39.456449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41324 (local address 127.31.87.130:33607)
0808 22:35:39.456549 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:39.456553 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:39.456564 (+ 11us) server_negotiation.cc:407] Connection header received
0808 22:35:39.456681 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:39.456684 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:39.456731 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:39.456816 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:35:39.456988 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:39.457530 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:39.458312 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:39.458507 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:39.458572 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:39.458776 (+ 204us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:35:39.459131 (+ 355us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:35:39.459214 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:39.459241 (+ 27us) server_negotiation.cc:299] Negotiation successful
0808 22:35:39.459294 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 22:35:39.464903 30282 raft_consensus.cc:1261] T bacf103945724ed894ed6751ce3ed61c P 13991b98219742f6ba4b13205a451857 [term 1 FOLLOWER]: Refusing update from remote peer 7746540492264da6aa1463613c1eff03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:35:39.464977 29844 raft_consensus.cc:1261] T bacf103945724ed894ed6751ce3ed61c P 3ba96d037bb24531b51e36ea04e355c8 [term 1 FOLLOWER]: Refusing update from remote peer 7746540492264da6aa1463613c1eff03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:35:39.465600 30385 consensus_queue.cc:1035] T bacf103945724ed894ed6751ce3ed61c P 7746540492264da6aa1463613c1eff03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13991b98219742f6ba4b13205a451857" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 40525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:35:39.465778 30389 consensus_queue.cc:1035] T bacf103945724ed894ed6751ce3ed61c P 7746540492264da6aa1463613c1eff03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ba96d037bb24531b51e36ea04e355c8" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 32959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:35:39.475728 30397 mvcc.cc:205] Tried to move back new op lower bound from 7316392097641684992 to 7316392097549201408. Current Snapshot: MvccSnapshot[applied={T|T < 7316392097641684992 or (T in {7316392097641684992})}]
I20260808 22:35:39.482998 29612 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42600:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:35:39.483168 29612 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42600:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:35:39.485651 29612 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 589100aff40a4ecbb12e02db3aa96ab1: Sending DeleteTablet for 3 replicas of tablet bacf103945724ed894ed6751ce3ed61c
I20260808 22:35:39.486496 30041 tablet_service.cc:1514] Processing DeleteTablet for tablet bacf103945724ed894ed6751ce3ed61c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:35:39 UTC) from {username='kudu', principal='kudu/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:41318
I20260808 22:35:39.486506 30257 tablet_service.cc:1514] Processing DeleteTablet for tablet bacf103945724ed894ed6751ce3ed61c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:35:39 UTC) from {username='kudu', principal='kudu/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:35314
I20260808 22:35:39.487075 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 29712
I20260808 22:35:39.487125 29814 tablet_service.cc:1514] Processing DeleteTablet for tablet bacf103945724ed894ed6751ce3ed61c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:35:39 UTC) from {username='kudu', principal='kudu/127.31.87.190@KRBTEST.COM'} at 127.0.0.1:45824
I20260808 22:35:39.489627 30408 tablet_replica.cc:330] T bacf103945724ed894ed6751ce3ed61c P 7746540492264da6aa1463613c1eff03: stopping tablet replica
I20260808 22:35:39.489923 30408 raft_consensus.cc:2229] T bacf103945724ed894ed6751ce3ed61c P 7746540492264da6aa1463613c1eff03 [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:35:39.490309 30408 raft_consensus.cc:2258] T bacf103945724ed894ed6751ce3ed61c P 7746540492264da6aa1463613c1eff03 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:35:39.490661 30409 tablet_replica.cc:330] T bacf103945724ed894ed6751ce3ed61c P 13991b98219742f6ba4b13205a451857: stopping tablet replica
I20260808 22:35:39.490916 30409 raft_consensus.cc:2229] T bacf103945724ed894ed6751ce3ed61c P 13991b98219742f6ba4b13205a451857 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:35:39.491178 30409 raft_consensus.cc:2258] T bacf103945724ed894ed6751ce3ed61c P 13991b98219742f6ba4b13205a451857 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:35:39.495007 30408 ts_tablet_manager.cc:1905] T bacf103945724ed894ed6751ce3ed61c P 7746540492264da6aa1463613c1eff03: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:35:39.495266 30409 ts_tablet_manager.cc:1905] T bacf103945724ed894ed6751ce3ed61c P 13991b98219742f6ba4b13205a451857: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:35:39.497718 30408 ts_tablet_manager.cc:1918] T bacf103945724ed894ed6751ce3ed61c P 7746540492264da6aa1463613c1eff03: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:35:39.497817 30408 log.cc:1196] T bacf103945724ed894ed6751ce3ed61c P 7746540492264da6aa1463613c1eff03: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/ts-1/wal/wals/bacf103945724ed894ed6751ce3ed61c
I20260808 22:35:39.498124 30408 ts_tablet_manager.cc:1939] T bacf103945724ed894ed6751ce3ed61c P 7746540492264da6aa1463613c1eff03: Deleting consensus metadata
I20260808 22:35:39.499078 30409 ts_tablet_manager.cc:1918] T bacf103945724ed894ed6751ce3ed61c P 13991b98219742f6ba4b13205a451857: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:35:39.499239 30409 log.cc:1196] T bacf103945724ed894ed6751ce3ed61c P 13991b98219742f6ba4b13205a451857: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786228291013016-32094-0/minicluster-data/ts-2/wal/wals/bacf103945724ed894ed6751ce3ed61c
I20260808 22:35:39.499266 29586 catalog_manager.cc:4841] TS 7746540492264da6aa1463613c1eff03 (127.31.87.130:33607): tablet bacf103945724ed894ed6751ce3ed61c (table test-table [id=151d7d088cb14d4fb9f3d6d00c0512ba]) successfully deleted
I20260808 22:35:39.499629 30409 ts_tablet_manager.cc:1939] T bacf103945724ed894ed6751ce3ed61c P 13991b98219742f6ba4b13205a451857: Deleting consensus metadata
I20260808 22:35:39.500777 29586 catalog_manager.cc:4841] TS 13991b98219742f6ba4b13205a451857 (127.31.87.131:40525): tablet bacf103945724ed894ed6751ce3ed61c (table test-table [id=151d7d088cb14d4fb9f3d6d00c0512ba]) successfully deleted
W20260808 22:35:39.511077 29587 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.87.129:32959 (error 108)
I20260808 22:35:39.511236 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 29935
I20260808 22:35:39.515738 30373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:39.515428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:32959 (local address 127.0.0.1:45826)
0808 22:35:39.515568 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:39.515648 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.129:32959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 22:35:39.515916 29587 catalog_manager.cc:4584] TS 3ba96d037bb24531b51e36ea04e355c8 (127.31.87.129:32959): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bacf103945724ed894ed6751ce3ed61c: Network error: Client connection negotiation failed: client connection to 127.31.87.129:32959: connect: Connection refused (error 111)
I20260808 22:35:39.520198 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 30157
I20260808 22:35:39.531183 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 29553
2026-08-08T22:35:39Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2297 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-08T22:35:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:35:39Z Disabled control of system clock
I20260808 22:35:39.562366 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:42789
--webserver_interface=127.31.87.190
--webserver_port=0
--builtin_ntp_servers=127.31.87.148:34761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:42789
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 22:35:39.671120 30415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:35:39.671365 30415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:35:39.671398 30415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:35:39.675232 30415 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:35:39.675278 30415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:35:39.675292 30415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:35:39.675307 30415 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:35:39.675326 30415 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:35:39.679966 30415 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:34761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:42789
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:42789
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.87.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30415
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:35:39.680907 30415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:35:39.682217 30415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:35:39.690135 30438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:39.690234 30432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:39.690194 30431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:39.690354 30415 server_base.cc:1034] running on GCE node
I20260808 22:35:39.690891 30415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:35:39.691874 30415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:35:39.693032 30415 hybrid_clock.cc:648] HybridClock initialized: now 1786228539693010 us; error 31 us; skew 500 ppm
I20260808 22:35:39.694833 30415 webserver.cc:460] Webserver started at http://127.31.87.190:44407/ using document root <none> and password file <none>
I20260808 22:35:39.695362 30415 fs_manager.cc:359] Metadata directory not provided
I20260808 22:35:39.695425 30415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:35:39.695631 30415 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:35:39.697654 30415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/master-0/data/instance:
uuid: "a59d475cd9ac4ff3a8f5f209452bd865"
format_stamp: "Formatted at 2026-08-08 22:35:39 on dist-test-slave-266d"
server_key: "2989f7fd0d345a9fe9c956ac82c6109a"
server_key_iv: "81e3637d039e68f07e5d8d8d59a42651"
server_key_version: "encryptionkey@0"
I20260808 22:35:39.698179 30415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/master-0/wal/instance:
uuid: "a59d475cd9ac4ff3a8f5f209452bd865"
format_stamp: "Formatted at 2026-08-08 22:35:39 on dist-test-slave-266d"
server_key: "2989f7fd0d345a9fe9c956ac82c6109a"
server_key_iv: "81e3637d039e68f07e5d8d8d59a42651"
server_key_version: "encryptionkey@0"
I20260808 22:35:39.701561 30415 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:35:39.703702 30446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:39.704799 30415 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:35:39.704907 30415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "a59d475cd9ac4ff3a8f5f209452bd865"
format_stamp: "Formatted at 2026-08-08 22:35:39 on dist-test-slave-266d"
server_key: "2989f7fd0d345a9fe9c956ac82c6109a"
server_key_iv: "81e3637d039e68f07e5d8d8d59a42651"
server_key_version: "encryptionkey@0"
I20260808 22:35:39.704993 30415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:35:39.714632 30415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:35:39.715166 30415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:35:39.715314 30415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:35:39.721812 30415 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.190:42789
I20260808 22:35:39.721822 30523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.190:42789 every 8 connection(s)
I20260808 22:35:39.722640 30415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:35:39.725473 30524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:35:39.727070 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 30415
I20260808 22:35:39.727200 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/master-0/wal/instance
I20260808 22:35:39.727456 32094 external_mini_cluster.cc:1428] Setting key 03a3ddd7271e70b5c3e37c86a8ec3ab0
I20260808 22:35:39.731375 30524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a59d475cd9ac4ff3a8f5f209452bd865: Bootstrap starting.
I20260808 22:35:39.733807 30524 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a59d475cd9ac4ff3a8f5f209452bd865: Neither blocks nor log segments found. Creating new log.
I20260808 22:35:39.734643 30524 log.cc:824] T 00000000000000000000000000000000 P a59d475cd9ac4ff3a8f5f209452bd865: Log is configured to *not* fsync() on all Append() calls
I20260808 22:35:39.736524 30524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a59d475cd9ac4ff3a8f5f209452bd865: No bootstrap required, opened a new log
I20260808 22:35:39.737040 30528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:39.728791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47820 (local address 127.31.87.190:42789)
0808 22:35:39.729443 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:39.729464 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:39.729507 (+ 43us) server_negotiation.cc:407] Connection header received
0808 22:35:39.730295 (+ 788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:39.730318 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:39.730777 (+ 459us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:39.731112 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:39.731524 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:39.732933 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:39.733836 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:39.734496 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:39.734649 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:39.735403 (+ 754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:39.735408 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:39.735420 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:35:39.735768 (+ 348us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:39.735820 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:35:39.735840 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:39.735915 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:39.736059 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 22:35:39.736327 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":100,"threads_started":1}
I20260808 22:35:39.739914 30524 raft_consensus.cc:348] T 00000000000000000000000000000000 P a59d475cd9ac4ff3a8f5f209452bd865 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a59d475cd9ac4ff3a8f5f209452bd865" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 42789 } }
I20260808 22:35:39.740108 30524 raft_consensus.cc:374] T 00000000000000000000000000000000 P a59d475cd9ac4ff3a8f5f209452bd865 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:35:39.740152 30524 raft_consensus.cc:729] T 00000000000000000000000000000000 P a59d475cd9ac4ff3a8f5f209452bd865 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a59d475cd9ac4ff3a8f5f209452bd865, State: Initialized, Role: FOLLOWER
I20260808 22:35:39.740526 30524 consensus_queue.cc:260] T 00000000000000000000000000000000 P a59d475cd9ac4ff3a8f5f209452bd865 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a59d475cd9ac4ff3a8f5f209452bd865" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 42789 } }
I20260808 22:35:39.740644 30524 raft_consensus.cc:388] T 00000000000000000000000000000000 P a59d475cd9ac4ff3a8f5f209452bd865 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:35:39.740705 30524 raft_consensus.cc:482] T 00000000000000000000000000000000 P a59d475cd9ac4ff3a8f5f209452bd865 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:35:39.740805 30524 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a59d475cd9ac4ff3a8f5f209452bd865 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:35:39.741664 30524 raft_consensus.cc:504] T 00000000000000000000000000000000 P a59d475cd9ac4ff3a8f5f209452bd865 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a59d475cd9ac4ff3a8f5f209452bd865" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 42789 } }
I20260808 22:35:39.741971 30524 leader_election.cc:302] T 00000000000000000000000000000000 P a59d475cd9ac4ff3a8f5f209452bd865 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a59d475cd9ac4ff3a8f5f209452bd865; no voters:
I20260808 22:35:39.742218 30524 leader_election.cc:288] T 00000000000000000000000000000000 P a59d475cd9ac4ff3a8f5f209452bd865 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:35:39.742333 30532 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a59d475cd9ac4ff3a8f5f209452bd865 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:35:39.742539 30532 raft_consensus.cc:686] T 00000000000000000000000000000000 P a59d475cd9ac4ff3a8f5f209452bd865 [term 1 LEADER]: Becoming Leader. State: Replica: a59d475cd9ac4ff3a8f5f209452bd865, State: Running, Role: LEADER
I20260808 22:35:39.742839 30532 consensus_queue.cc:237] T 00000000000000000000000000000000 P a59d475cd9ac4ff3a8f5f209452bd865 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a59d475cd9ac4ff3a8f5f209452bd865" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 42789 } }
I20260808 22:35:39.743342 30524 sys_catalog.cc:564] T 00000000000000000000000000000000 P a59d475cd9ac4ff3a8f5f209452bd865 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:35:39.743834 30534 sys_catalog.cc:455] T 00000000000000000000000000000000 P a59d475cd9ac4ff3a8f5f209452bd865 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a59d475cd9ac4ff3a8f5f209452bd865. Latest consensus state: current_term: 1 leader_uuid: "a59d475cd9ac4ff3a8f5f209452bd865" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a59d475cd9ac4ff3a8f5f209452bd865" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 42789 } } }
I20260808 22:35:39.743961 30534 sys_catalog.cc:458] T 00000000000000000000000000000000 P a59d475cd9ac4ff3a8f5f209452bd865 [sys.catalog]: This master's current role is: LEADER
I20260808 22:35:39.744251 30533 sys_catalog.cc:455] T 00000000000000000000000000000000 P a59d475cd9ac4ff3a8f5f209452bd865 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a59d475cd9ac4ff3a8f5f209452bd865" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a59d475cd9ac4ff3a8f5f209452bd865" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 42789 } } }
I20260808 22:35:39.744342 30533 sys_catalog.cc:458] T 00000000000000000000000000000000 P a59d475cd9ac4ff3a8f5f209452bd865 [sys.catalog]: This master's current role is: LEADER
I20260808 22:35:39.744374 30539 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:35:39.747200 30539 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:35:39.752210 30539 catalog_manager.cc:1347] Generated new cluster ID: 118c12d76d89454abcf71d31436d45dd
I20260808 22:35:39.752287 30539 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:35:39.767477 30539 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:35:39.768348 30539 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:35:39.778468 30539 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a59d475cd9ac4ff3a8f5f209452bd865: Generated new TSK 0
I20260808 22:35:39.779066 30539 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:35:39.790918 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.129:0
--local_ip_for_outbound_sockets=127.31.87.129
--webserver_interface=127.31.87.129
--webserver_port=0
--tserver_master_addrs=127.31.87.190:42789
--builtin_ntp_servers=127.31.87.148:34761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 22:35:39.901623 30559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:35:39.901859 30559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:35:39.901896 30559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:35:39.905809 30559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:35:39.905864 30559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:35:39.905926 30559 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.129
I20260808 22:35:39.910717 30559 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:34761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.87.129
--webserver_port=0
--tserver_master_addrs=127.31.87.190:42789
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30559
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.129
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:35:39.911700 30559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:35:39.913035 30559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:35:39.922186 30568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:39.922195 30569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:39.922286 30559 server_base.cc:1034] running on GCE node
W20260808 22:35:39.922235 30571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:39.922716 30559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:35:39.923308 30559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:35:39.924464 30559 hybrid_clock.cc:648] HybridClock initialized: now 1786228539924448 us; error 29 us; skew 500 ppm
I20260808 22:35:39.926156 30559 webserver.cc:460] Webserver started at http://127.31.87.129:37865/ using document root <none> and password file <none>
I20260808 22:35:39.926679 30559 fs_manager.cc:359] Metadata directory not provided
I20260808 22:35:39.926746 30559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:35:39.926956 30559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:35:39.928874 30559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/ts-0/data/instance:
uuid: "9c227810e58d448098cfb7b2f5857077"
format_stamp: "Formatted at 2026-08-08 22:35:39 on dist-test-slave-266d"
server_key: "db757ed493d73553db2219fa1a58359b"
server_key_iv: "3ebbbb9dc43896a5115d2829bee31d89"
server_key_version: "encryptionkey@0"
I20260808 22:35:39.929426 30559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/ts-0/wal/instance:
uuid: "9c227810e58d448098cfb7b2f5857077"
format_stamp: "Formatted at 2026-08-08 22:35:39 on dist-test-slave-266d"
server_key: "db757ed493d73553db2219fa1a58359b"
server_key_iv: "3ebbbb9dc43896a5115d2829bee31d89"
server_key_version: "encryptionkey@0"
I20260808 22:35:39.932713 30559 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 22:35:39.934933 30578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:39.935981 30559 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 22:35:39.936085 30559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/ts-0/wal
uuid: "9c227810e58d448098cfb7b2f5857077"
format_stamp: "Formatted at 2026-08-08 22:35:39 on dist-test-slave-266d"
server_key: "db757ed493d73553db2219fa1a58359b"
server_key_iv: "3ebbbb9dc43896a5115d2829bee31d89"
server_key_version: "encryptionkey@0"
I20260808 22:35:39.936179 30559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:35:39.966259 30559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:35:39.966902 30559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:35:39.967074 30559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:35:39.967636 30559 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:35:39.968526 30559 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:35:39.968572 30559 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:39.968624 30559 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:35:39.968654 30559 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:39.976697 30559 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.129:42683
I20260808 22:35:39.976724 30749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.129:42683 every 8 connection(s)
I20260808 22:35:39.977545 30559 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/ts-0/data/info.pb
I20260808 22:35:39.977707 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 30559
I20260808 22:35:39.977833 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/ts-0/wal/instance
I20260808 22:35:39.978163 32094 external_mini_cluster.cc:1428] Setting key f15f54feb9fd1f79f10833d030721fb1
I20260808 22:35:39.980139 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.130:0
--local_ip_for_outbound_sockets=127.31.87.130
--webserver_interface=127.31.87.130
--webserver_port=0
--tserver_master_addrs=127.31.87.190:42789
--builtin_ntp_servers=127.31.87.148:34761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 22:35:39.987270 30528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:39.979233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:44307 (local address 127.31.87.190:42789)
0808 22:35:39.979372 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:39.979377 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:39.980578 (+ 1201us) server_negotiation.cc:407] Connection header received
0808 22:35:39.982167 (+ 1589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:39.982171 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:39.982224 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:39.982299 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:39.983222 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:39.983745 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:39.985407 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:39.985626 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:39.985704 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:39.986842 (+ 1138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:39.986845 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:39.986847 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:35:39.986860 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:39.986880 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:35:39.986888 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:39.986943 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:39.987099 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 22:35:39.987157 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 22:35:39.988373 30752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:39.979725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:42789 (local address 127.31.87.129:44307)
0808 22:35:39.980425 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:39.980476 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 22:35:39.981970 (+ 1494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:35:39.982471 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:35:39.982478 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:35:39.982562 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:35:39.983081 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:39.983100 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:39.983879 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:39.983882 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:39.985285 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:39.985298 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:39.985769 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:39.985775 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:39.985967 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:39.986555 (+ 588us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:35:39.986558 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:35:39.986612 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:35:39.986694 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:35:39.986963 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:35:39.986971 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:35:39.986978 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 22:35:39.987107 (+ 129us) client_negotiation.cc:240] Negotiation successful
0808 22:35:39.987397 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":116,"threads_started":1}
I20260808 22:35:39.989502 30750 heartbeater.cc:347] Connected to a master server at 127.31.87.190:42789
I20260808 22:35:39.989739 30750 heartbeater.cc:464] Registering TS with master...
I20260808 22:35:39.990334 30750 heartbeater.cc:511] Master 127.31.87.190:42789 requested a full tablet report, sending...
I20260808 22:35:39.991796 30463 ts_manager.cc:194] Registered new tserver with Master: 9c227810e58d448098cfb7b2f5857077 (127.31.87.129:42683)
I20260808 22:35:39.992879 30463 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.87.129:44307
W20260808 22:35:40.092131 30753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:35:40.092386 30753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:35:40.092445 30753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:35:40.096438 30753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:35:40.096494 30753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:35:40.096580 30753 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.130
I20260808 22:35:40.101249 30753 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:34761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.87.130
--webserver_port=0
--tserver_master_addrs=127.31.87.190:42789
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30753
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.130
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:35:40.102247 30753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:35:40.103595 30753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:35:40.112968 30766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:40.112994 30761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:40.112994 30762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:40.113209 30753 server_base.cc:1034] running on GCE node
I20260808 22:35:40.113663 30753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:35:40.114233 30753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:35:40.115374 30753 hybrid_clock.cc:648] HybridClock initialized: now 1786228540115352 us; error 37 us; skew 500 ppm
I20260808 22:35:40.117120 30753 webserver.cc:460] Webserver started at http://127.31.87.130:43303/ using document root <none> and password file <none>
I20260808 22:35:40.117668 30753 fs_manager.cc:359] Metadata directory not provided
I20260808 22:35:40.117728 30753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:35:40.117938 30753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:35:40.120018 30753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/ts-1/data/instance:
uuid: "27b5a31c0cf54826abbb5628302f41f7"
format_stamp: "Formatted at 2026-08-08 22:35:40 on dist-test-slave-266d"
server_key: "3a6af7260917d0bfb9c81e0321fcc371"
server_key_iv: "1d7cab9f6b96a75c2c8810d7fe8fb973"
server_key_version: "encryptionkey@0"
I20260808 22:35:40.120530 30753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/ts-1/wal/instance:
uuid: "27b5a31c0cf54826abbb5628302f41f7"
format_stamp: "Formatted at 2026-08-08 22:35:40 on dist-test-slave-266d"
server_key: "3a6af7260917d0bfb9c81e0321fcc371"
server_key_iv: "1d7cab9f6b96a75c2c8810d7fe8fb973"
server_key_version: "encryptionkey@0"
I20260808 22:35:40.123977 30753 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 22:35:40.126262 30775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:40.127374 30753 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:35:40.127475 30753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/ts-1/wal
uuid: "27b5a31c0cf54826abbb5628302f41f7"
format_stamp: "Formatted at 2026-08-08 22:35:40 on dist-test-slave-266d"
server_key: "3a6af7260917d0bfb9c81e0321fcc371"
server_key_iv: "1d7cab9f6b96a75c2c8810d7fe8fb973"
server_key_version: "encryptionkey@0"
I20260808 22:35:40.127561 30753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:35:40.140983 30753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:35:40.141569 30753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:35:40.141719 30753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:35:40.142246 30753 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:35:40.143090 30753 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:35:40.143136 30753 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:40.143184 30753 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:35:40.143213 30753 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:40.151291 30753 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.130:39771
I20260808 22:35:40.151314 30935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.130:39771 every 8 connection(s)
I20260808 22:35:40.152117 30753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/ts-1/data/info.pb
I20260808 22:35:40.156801 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 30753
I20260808 22:35:40.156915 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/ts-1/wal/instance
I20260808 22:35:40.157238 32094 external_mini_cluster.cc:1428] Setting key 1040dd0c233dfa9593e234290bd6e95b
I20260808 22:35:40.159339 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.87.131:0
--local_ip_for_outbound_sockets=127.31.87.131
--webserver_interface=127.31.87.131
--webserver_port=0
--tserver_master_addrs=127.31.87.190:42789
--builtin_ntp_servers=127.31.87.148:34761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 22:35:40.161310 30528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:40.153811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.130:55511 (local address 127.31.87.190:42789)
0808 22:35:40.153947 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:40.153950 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:40.154897 (+ 947us) server_negotiation.cc:407] Connection header received
0808 22:35:40.156208 (+ 1311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:40.156210 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:40.156255 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:40.156342 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:40.157229 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:40.157920 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:40.159307 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:40.159633 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:40.159755 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:40.160784 (+ 1029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:40.160789 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:40.160792 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:35:40.160809 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:40.160833 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:35:40.160845 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:40.160930 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:40.161044 (+ 114us) server_negotiation.cc:299] Negotiation successful
0808 22:35:40.161146 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 22:35:40.162250 30941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:40.154169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:42789 (local address 127.31.87.130:55511)
0808 22:35:40.154779 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:40.154817 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 22:35:40.156030 (+ 1213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:35:40.156468 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:35:40.156475 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:35:40.156552 (+ 77us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:35:40.157059 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:40.157075 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:40.158047 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:40.158050 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:40.159153 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:40.159164 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:40.159787 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:40.159790 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:40.159934 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:40.160497 (+ 563us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:35:40.160499 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:35:40.160537 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:35:40.160598 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:35:40.160925 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:35:40.160929 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:35:40.160933 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:35:40.161016 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 22:35:40.161320 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":115,"threads_started":1}
I20260808 22:35:40.163344 30937 heartbeater.cc:347] Connected to a master server at 127.31.87.190:42789
I20260808 22:35:40.163558 30937 heartbeater.cc:464] Registering TS with master...
I20260808 22:35:40.164013 30937 heartbeater.cc:511] Master 127.31.87.190:42789 requested a full tablet report, sending...
I20260808 22:35:40.165035 30463 ts_manager.cc:194] Registered new tserver with Master: 27b5a31c0cf54826abbb5628302f41f7 (127.31.87.130:39771)
I20260808 22:35:40.165836 30463 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.87.130:55511
W20260808 22:35:40.270797 30944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:35:40.271031 30944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:35:40.271077 30944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:35:40.274884 30944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:35:40.274930 30944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:35:40.274982 30944 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.87.131
I20260808 22:35:40.279610 30944 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:34761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.87.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.87.131
--webserver_port=0
--tserver_master_addrs=127.31.87.190:42789
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30944
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.87.131
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:35:40.280584 30944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:35:40.281883 30944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:35:40.291091 30955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:40.291097 30950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:40.291297 30944 server_base.cc:1034] running on GCE node
W20260808 22:35:40.291500 30951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:40.291895 30944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:35:40.292492 30944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:35:40.293650 30944 hybrid_clock.cc:648] HybridClock initialized: now 1786228540293630 us; error 30 us; skew 500 ppm
I20260808 22:35:40.295336 30944 webserver.cc:460] Webserver started at http://127.31.87.131:43769/ using document root <none> and password file <none>
I20260808 22:35:40.295892 30944 fs_manager.cc:359] Metadata directory not provided
I20260808 22:35:40.295961 30944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:35:40.296135 30944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:35:40.298118 30944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/ts-2/data/instance:
uuid: "4cb8a1f6b5ac48fa8a568998b3cd853f"
format_stamp: "Formatted at 2026-08-08 22:35:40 on dist-test-slave-266d"
server_key: "e16280c06d63b79a9331c17e1c43f152"
server_key_iv: "aec7386c68a3408530ec93f7c93e4f3d"
server_key_version: "encryptionkey@0"
I20260808 22:35:40.298626 30944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/ts-2/wal/instance:
uuid: "4cb8a1f6b5ac48fa8a568998b3cd853f"
format_stamp: "Formatted at 2026-08-08 22:35:40 on dist-test-slave-266d"
server_key: "e16280c06d63b79a9331c17e1c43f152"
server_key_iv: "aec7386c68a3408530ec93f7c93e4f3d"
server_key_version: "encryptionkey@0"
I20260808 22:35:40.302029 30944 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:35:40.304214 30965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:40.305358 30944 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:35:40.305450 30944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/ts-2/wal
uuid: "4cb8a1f6b5ac48fa8a568998b3cd853f"
format_stamp: "Formatted at 2026-08-08 22:35:40 on dist-test-slave-266d"
server_key: "e16280c06d63b79a9331c17e1c43f152"
server_key_iv: "aec7386c68a3408530ec93f7c93e4f3d"
server_key_version: "encryptionkey@0"
I20260808 22:35:40.305527 30944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:35:40.314091 30944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:35:40.314602 30944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:35:40.314728 30944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:35:40.315176 30944 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:35:40.315980 30944 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:35:40.316030 30944 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:40.316071 30944 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:35:40.316092 30944 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:40.324309 30944 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.131:44517
I20260808 22:35:40.324331 31160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.131:44517 every 8 connection(s)
I20260808 22:35:40.325070 30944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/ts-2/data/info.pb
I20260808 22:35:40.326429 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 30944
I20260808 22:35:40.326541 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/ts-2/wal/instance
I20260808 22:35:40.326854 32094 external_mini_cluster.cc:1428] Setting key cb48aaea47499db0b91beb543669db78
I20260808 22:35:40.333341 30528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:40.326639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.131:56503 (local address 127.31.87.190:42789)
0808 22:35:40.326780 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:40.326785 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:40.327874 (+ 1089us) server_negotiation.cc:407] Connection header received
0808 22:35:40.329278 (+ 1404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:40.329280 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:40.329326 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:40.329390 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:40.330138 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:40.330640 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:40.331745 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:40.331957 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:40.332030 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:40.332955 (+ 925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:40.332957 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:40.332960 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:35:40.332971 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:40.332990 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:35:40.332997 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:40.333051 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:40.333185 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 22:35:40.333232 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 22:35:40.334054 31163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:40.327031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:42789 (local address 127.31.87.131:56503)
0808 22:35:40.327727 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:40.327775 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 22:35:40.329079 (+ 1304us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:35:40.329525 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:35:40.329532 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:35:40.329608 (+ 76us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:35:40.330036 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:40.330048 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:40.330755 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:40.330758 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:40.331650 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:40.331660 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:40.332076 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:40.332079 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:40.332213 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:40.332728 (+ 515us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:35:40.332730 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:35:40.332767 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:35:40.332826 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:35:40.333057 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:35:40.333061 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:35:40.333065 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:35:40.333178 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 22:35:40.333384 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":108,"threads_started":1}
I20260808 22:35:40.335035 31161 heartbeater.cc:347] Connected to a master server at 127.31.87.190:42789
I20260808 22:35:40.335214 31161 heartbeater.cc:464] Registering TS with master...
I20260808 22:35:40.335628 31161 heartbeater.cc:511] Master 127.31.87.190:42789 requested a full tablet report, sending...
I20260808 22:35:40.336505 30463 ts_manager.cc:194] Registered new tserver with Master: 4cb8a1f6b5ac48fa8a568998b3cd853f (127.31.87.131:44517)
I20260808 22:35:40.337033 30463 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.87.131:56503
I20260808 22:35:40.340404 32094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:35:40.345614 30528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:40.342142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49140 (local address 127.31.87.190:42789)
0808 22:35:40.342268 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:40.342272 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:40.342427 (+ 155us) server_negotiation.cc:407] Connection header received
0808 22:35:40.342529 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:40.342531 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:40.342567 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:40.342648 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:40.342895 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:40.343373 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:40.344269 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:40.344477 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:40.344550 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:40.345263 (+ 713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:40.345266 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:40.345267 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:35:40.345278 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:40.345294 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:35:40.345302 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:40.345355 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:40.345420 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 22:35:40.345463 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 22:35:40.348122 30463 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49140:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:35:40.351086 30463 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:35:40.364542 31186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:40.359545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:42683 (local address 127.0.0.1:33362)
0808 22:35:40.360255 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:40.360268 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:35:40.360345 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:35:40.360804 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:35:40.360807 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:35:40.360823 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:35:40.360895 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:40.360900 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:40.362229 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:40.362231 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:40.363216 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:40.363226 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:40.364022 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:40.364026 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:40.364140 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:40.364177 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 22:35:40.364280 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 22:35:40.364396 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":651,"mutex_wait_us":19,"thread_start_us":87,"threads_started":1}
I20260808 22:35:40.364953 31187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:40.359947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33362 (local address 127.31.87.129:42683)
0808 22:35:40.360369 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:40.360380 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:40.360402 (+ 22us) server_negotiation.cc:407] Connection header received
0808 22:35:40.360469 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:40.360474 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:40.360638 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:40.360775 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:35:40.360993 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:40.362115 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:40.363350 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:40.363901 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:40.363996 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:40.364185 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:40.364736 (+ 551us) server_negotiation.cc:299] Negotiation successful
0808 22:35:40.364835 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":131,"threads_started":1}
I20260808 22:35:40.364957 31181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:40.359003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55070 (local address 127.31.87.131:44517)
0808 22:35:40.360081 (+ 1078us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:40.360093 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:40.360128 (+ 35us) server_negotiation.cc:407] Connection header received
0808 22:35:40.360205 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:40.360210 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:40.360395 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:40.360630 (+ 235us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:35:40.360895 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:40.362088 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:40.363465 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:40.364176 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:40.364287 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:40.364444 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:40.364571 (+ 127us) server_negotiation.cc:299] Negotiation successful
0808 22:35:40.364813 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":976,"thread_start_us":138,"threads_started":1}
I20260808 22:35:40.364760 31179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:40.358884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.131:44517 (local address 127.0.0.1:55070)
0808 22:35:40.359491 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:40.359537 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 22:35:40.359653 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:35:40.360537 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:35:40.360540 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:35:40.360554 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:35:40.360752 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:40.360760 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:40.362220 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:40.362223 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:40.363352 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:40.363361 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:40.364278 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:40.364281 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:40.364439 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:40.364462 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 22:35:40.364532 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 22:35:40.364595 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"mutex_wait_us":134,"thread_start_us":78,"threads_started":1}
I20260808 22:35:40.365074 31185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:40.359366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60264 (local address 127.31.87.130:39771)
0808 22:35:40.359963 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:40.359974 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:40.359998 (+ 24us) server_negotiation.cc:407] Connection header received
0808 22:35:40.360077 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:40.360083 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:40.360286 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:40.360438 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:35:40.360791 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:40.361645 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:40.363207 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:40.363945 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:40.364052 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:40.364227 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:40.364841 (+ 614us) server_negotiation.cc:299] Negotiation successful
0808 22:35:40.364933 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":108,"threads_started":1}
I20260808 22:35:40.364776 31180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:40.359019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.130:39771 (local address 127.0.0.1:60264)
0808 22:35:40.359747 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:40.359759 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:35:40.359840 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:35:40.360464 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:35:40.360469 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:35:40.360491 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:35:40.360680 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:40.360688 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:40.361751 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:40.361754 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:40.363078 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:40.363089 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:40.364292 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:40.364295 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:40.364453 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:40.364478 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 22:35:40.364545 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 22:35:40.364617 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"mutex_wait_us":46,"thread_start_us":120,"threads_started":1}
I20260808 22:35:40.367681 31038 tablet_service.cc:1467] Processing CreateTablet for tablet 4ba16ab551a54de4862cfb1345a2af21 (DEFAULT_TABLE table=test-table [id=fb94ac81aa334832a9665a0967a0047f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:35:40.367686 30850 tablet_service.cc:1467] Processing CreateTablet for tablet 4ba16ab551a54de4862cfb1345a2af21 (DEFAULT_TABLE table=test-table [id=fb94ac81aa334832a9665a0967a0047f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:35:40.368574 31038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ba16ab551a54de4862cfb1345a2af21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:35:40.368574 30850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ba16ab551a54de4862cfb1345a2af21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:35:40.370497 30651 tablet_service.cc:1467] Processing CreateTablet for tablet 4ba16ab551a54de4862cfb1345a2af21 (DEFAULT_TABLE table=test-table [id=fb94ac81aa334832a9665a0967a0047f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:35:40.371456 30651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ba16ab551a54de4862cfb1345a2af21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:35:40.375150 31188 tablet_bootstrap.cc:492] T 4ba16ab551a54de4862cfb1345a2af21 P 4cb8a1f6b5ac48fa8a568998b3cd853f: Bootstrap starting.
I20260808 22:35:40.375689 31190 tablet_bootstrap.cc:492] T 4ba16ab551a54de4862cfb1345a2af21 P 27b5a31c0cf54826abbb5628302f41f7: Bootstrap starting.
I20260808 22:35:40.377542 31188 tablet_bootstrap.cc:654] T 4ba16ab551a54de4862cfb1345a2af21 P 4cb8a1f6b5ac48fa8a568998b3cd853f: Neither blocks nor log segments found. Creating new log.
I20260808 22:35:40.377728 31193 tablet_bootstrap.cc:492] T 4ba16ab551a54de4862cfb1345a2af21 P 9c227810e58d448098cfb7b2f5857077: Bootstrap starting.
I20260808 22:35:40.377766 31190 tablet_bootstrap.cc:654] T 4ba16ab551a54de4862cfb1345a2af21 P 27b5a31c0cf54826abbb5628302f41f7: Neither blocks nor log segments found. Creating new log.
I20260808 22:35:40.378559 31188 log.cc:824] T 4ba16ab551a54de4862cfb1345a2af21 P 4cb8a1f6b5ac48fa8a568998b3cd853f: Log is configured to *not* fsync() on all Append() calls
I20260808 22:35:40.378669 31190 log.cc:824] T 4ba16ab551a54de4862cfb1345a2af21 P 27b5a31c0cf54826abbb5628302f41f7: Log is configured to *not* fsync() on all Append() calls
I20260808 22:35:40.379595 31193 tablet_bootstrap.cc:654] T 4ba16ab551a54de4862cfb1345a2af21 P 9c227810e58d448098cfb7b2f5857077: Neither blocks nor log segments found. Creating new log.
I20260808 22:35:40.380235 31193 log.cc:824] T 4ba16ab551a54de4862cfb1345a2af21 P 9c227810e58d448098cfb7b2f5857077: Log is configured to *not* fsync() on all Append() calls
I20260808 22:35:40.380523 31188 tablet_bootstrap.cc:492] T 4ba16ab551a54de4862cfb1345a2af21 P 4cb8a1f6b5ac48fa8a568998b3cd853f: No bootstrap required, opened a new log
I20260808 22:35:40.380523 31190 tablet_bootstrap.cc:492] T 4ba16ab551a54de4862cfb1345a2af21 P 27b5a31c0cf54826abbb5628302f41f7: No bootstrap required, opened a new log
I20260808 22:35:40.380712 31188 ts_tablet_manager.cc:1397] T 4ba16ab551a54de4862cfb1345a2af21 P 4cb8a1f6b5ac48fa8a568998b3cd853f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 22:35:40.380712 31190 ts_tablet_manager.cc:1397] T 4ba16ab551a54de4862cfb1345a2af21 P 27b5a31c0cf54826abbb5628302f41f7: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 22:35:40.381970 31193 tablet_bootstrap.cc:492] T 4ba16ab551a54de4862cfb1345a2af21 P 9c227810e58d448098cfb7b2f5857077: No bootstrap required, opened a new log
I20260808 22:35:40.382144 31193 ts_tablet_manager.cc:1397] T 4ba16ab551a54de4862cfb1345a2af21 P 9c227810e58d448098cfb7b2f5857077: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 22:35:40.384902 31190 raft_consensus.cc:348] T 4ba16ab551a54de4862cfb1345a2af21 P 27b5a31c0cf54826abbb5628302f41f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb8a1f6b5ac48fa8a568998b3cd853f" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 44517 } } peers { permanent_uuid: "27b5a31c0cf54826abbb5628302f41f7" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 39771 } } peers { permanent_uuid: "9c227810e58d448098cfb7b2f5857077" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 42683 } }
I20260808 22:35:40.384902 31188 raft_consensus.cc:348] T 4ba16ab551a54de4862cfb1345a2af21 P 4cb8a1f6b5ac48fa8a568998b3cd853f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb8a1f6b5ac48fa8a568998b3cd853f" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 44517 } } peers { permanent_uuid: "27b5a31c0cf54826abbb5628302f41f7" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 39771 } } peers { permanent_uuid: "9c227810e58d448098cfb7b2f5857077" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 42683 } }
I20260808 22:35:40.385187 31190 raft_consensus.cc:374] T 4ba16ab551a54de4862cfb1345a2af21 P 27b5a31c0cf54826abbb5628302f41f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:35:40.385187 31188 raft_consensus.cc:374] T 4ba16ab551a54de4862cfb1345a2af21 P 4cb8a1f6b5ac48fa8a568998b3cd853f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:35:40.385036 31193 raft_consensus.cc:348] T 4ba16ab551a54de4862cfb1345a2af21 P 9c227810e58d448098cfb7b2f5857077 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb8a1f6b5ac48fa8a568998b3cd853f" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 44517 } } peers { permanent_uuid: "27b5a31c0cf54826abbb5628302f41f7" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 39771 } } peers { permanent_uuid: "9c227810e58d448098cfb7b2f5857077" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 42683 } }
I20260808 22:35:40.385250 31190 raft_consensus.cc:729] T 4ba16ab551a54de4862cfb1345a2af21 P 27b5a31c0cf54826abbb5628302f41f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27b5a31c0cf54826abbb5628302f41f7, State: Initialized, Role: FOLLOWER
I20260808 22:35:40.385239 31188 raft_consensus.cc:729] T 4ba16ab551a54de4862cfb1345a2af21 P 4cb8a1f6b5ac48fa8a568998b3cd853f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cb8a1f6b5ac48fa8a568998b3cd853f, State: Initialized, Role: FOLLOWER
I20260808 22:35:40.385290 31193 raft_consensus.cc:374] T 4ba16ab551a54de4862cfb1345a2af21 P 9c227810e58d448098cfb7b2f5857077 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:35:40.385341 31193 raft_consensus.cc:729] T 4ba16ab551a54de4862cfb1345a2af21 P 9c227810e58d448098cfb7b2f5857077 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c227810e58d448098cfb7b2f5857077, State: Initialized, Role: FOLLOWER
I20260808 22:35:40.385740 31188 consensus_queue.cc:260] T 4ba16ab551a54de4862cfb1345a2af21 P 4cb8a1f6b5ac48fa8a568998b3cd853f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb8a1f6b5ac48fa8a568998b3cd853f" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 44517 } } peers { permanent_uuid: "27b5a31c0cf54826abbb5628302f41f7" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 39771 } } peers { permanent_uuid: "9c227810e58d448098cfb7b2f5857077" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 42683 } }
I20260808 22:35:40.385740 31190 consensus_queue.cc:260] T 4ba16ab551a54de4862cfb1345a2af21 P 27b5a31c0cf54826abbb5628302f41f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb8a1f6b5ac48fa8a568998b3cd853f" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 44517 } } peers { permanent_uuid: "27b5a31c0cf54826abbb5628302f41f7" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 39771 } } peers { permanent_uuid: "9c227810e58d448098cfb7b2f5857077" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 42683 } }
I20260808 22:35:40.385779 31193 consensus_queue.cc:260] T 4ba16ab551a54de4862cfb1345a2af21 P 9c227810e58d448098cfb7b2f5857077 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb8a1f6b5ac48fa8a568998b3cd853f" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 44517 } } peers { permanent_uuid: "27b5a31c0cf54826abbb5628302f41f7" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 39771 } } peers { permanent_uuid: "9c227810e58d448098cfb7b2f5857077" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 42683 } }
I20260808 22:35:40.386423 30937 heartbeater.cc:503] Master 127.31.87.190:42789 was elected leader, sending a full tablet report...
I20260808 22:35:40.386737 31190 ts_tablet_manager.cc:1428] T 4ba16ab551a54de4862cfb1345a2af21 P 27b5a31c0cf54826abbb5628302f41f7: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 22:35:40.386751 31193 ts_tablet_manager.cc:1428] T 4ba16ab551a54de4862cfb1345a2af21 P 9c227810e58d448098cfb7b2f5857077: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 22:35:40.386986 30750 heartbeater.cc:503] Master 127.31.87.190:42789 was elected leader, sending a full tablet report...
I20260808 22:35:40.387013 31161 heartbeater.cc:503] Master 127.31.87.190:42789 was elected leader, sending a full tablet report...
I20260808 22:35:40.388166 31188 ts_tablet_manager.cc:1428] T 4ba16ab551a54de4862cfb1345a2af21 P 4cb8a1f6b5ac48fa8a568998b3cd853f: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
W20260808 22:35:40.402957 30940 tablet.cc:2369] T 4ba16ab551a54de4862cfb1345a2af21 P 27b5a31c0cf54826abbb5628302f41f7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:35:40.457108 31202 raft_consensus.cc:482] T 4ba16ab551a54de4862cfb1345a2af21 P 9c227810e58d448098cfb7b2f5857077 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:35:40.457340 31202 raft_consensus.cc:504] T 4ba16ab551a54de4862cfb1345a2af21 P 9c227810e58d448098cfb7b2f5857077 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb8a1f6b5ac48fa8a568998b3cd853f" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 44517 } } peers { permanent_uuid: "27b5a31c0cf54826abbb5628302f41f7" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 39771 } } peers { permanent_uuid: "9c227810e58d448098cfb7b2f5857077" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 42683 } }
I20260808 22:35:40.458649 31202 leader_election.cc:288] T 4ba16ab551a54de4862cfb1345a2af21 P 9c227810e58d448098cfb7b2f5857077 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4cb8a1f6b5ac48fa8a568998b3cd853f (127.31.87.131:44517), 27b5a31c0cf54826abbb5628302f41f7 (127.31.87.130:39771)
I20260808 22:35:40.462438 31181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:40.459025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:55445 (local address 127.31.87.131:44517)
0808 22:35:40.459155 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:40.459159 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:40.459339 (+ 180us) server_negotiation.cc:407] Connection header received
0808 22:35:40.459495 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:40.459498 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:40.459538 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:40.459604 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:35:40.459898 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:40.460398 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:40.461509 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:40.461883 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:40.461987 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:40.462039 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:40.462239 (+ 200us) server_negotiation.cc:299] Negotiation successful
0808 22:35:40.462319 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 22:35:40.462437 31185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:40.458993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.87.129:59015 (local address 127.31.87.130:39771)
0808 22:35:40.459121 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:40.459126 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:40.459344 (+ 218us) server_negotiation.cc:407] Connection header received
0808 22:35:40.459478 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:40.459481 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:40.459527 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:40.459591 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:35:40.459914 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:40.460395 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:40.461495 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:40.461886 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:40.461990 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:40.462039 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:40.462234 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 22:35:40.462305 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 22:35:40.462450 31204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:40.458875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.131:44517 (local address 127.31.87.129:55445)
0808 22:35:40.459253 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:40.459265 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:35:40.459386 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:35:40.459622 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:35:40.459625 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:35:40.459640 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:35:40.459792 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:40.459797 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:40.460497 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:40.460501 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:40.461391 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:40.461400 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:40.461989 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:40.461992 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:40.462091 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:40.462143 (+ 52us) client_negotiation.cc:769] Sending connection context
0808 22:35:40.462209 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 22:35:40.462313 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"mutex_wait_us":90,"spinlock_wait_cycles":8576}
I20260808 22:35:40.462450 30752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:40.458881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.130:39771 (local address 127.31.87.129:59015)
0808 22:35:40.459266 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:40.459278 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:35:40.459383 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:35:40.459611 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:35:40.459614 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:35:40.459628 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:35:40.459804 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:40.459810 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:40.460498 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:40.460501 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:40.461381 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:35:40.461390 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:35:40.462010 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:35:40.462014 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:35:40.462113 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:40.462146 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 22:35:40.462209 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 22:35:40.462319 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"mutex_wait_us":127,"spinlock_wait_cycles":25728,"thread_start_us":115,"threads_started":1}
I20260808 22:35:40.463261 30877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ba16ab551a54de4862cfb1345a2af21" candidate_uuid: "9c227810e58d448098cfb7b2f5857077" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27b5a31c0cf54826abbb5628302f41f7" is_pre_election: true
I20260808 22:35:40.463263 31073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ba16ab551a54de4862cfb1345a2af21" candidate_uuid: "9c227810e58d448098cfb7b2f5857077" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cb8a1f6b5ac48fa8a568998b3cd853f" is_pre_election: true
I20260808 22:35:40.463577 31073 raft_consensus.cc:2454] T 4ba16ab551a54de4862cfb1345a2af21 P 4cb8a1f6b5ac48fa8a568998b3cd853f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c227810e58d448098cfb7b2f5857077 in term 0.
I20260808 22:35:40.463575 30877 raft_consensus.cc:2454] T 4ba16ab551a54de4862cfb1345a2af21 P 27b5a31c0cf54826abbb5628302f41f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c227810e58d448098cfb7b2f5857077 in term 0.
I20260808 22:35:40.464073 30581 leader_election.cc:302] T 4ba16ab551a54de4862cfb1345a2af21 P 9c227810e58d448098cfb7b2f5857077 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cb8a1f6b5ac48fa8a568998b3cd853f, 9c227810e58d448098cfb7b2f5857077; no voters:
I20260808 22:35:40.464352 31202 raft_consensus.cc:2781] T 4ba16ab551a54de4862cfb1345a2af21 P 9c227810e58d448098cfb7b2f5857077 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:35:40.464434 31202 raft_consensus.cc:482] T 4ba16ab551a54de4862cfb1345a2af21 P 9c227810e58d448098cfb7b2f5857077 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:35:40.464464 31202 raft_consensus.cc:3037] T 4ba16ab551a54de4862cfb1345a2af21 P 9c227810e58d448098cfb7b2f5857077 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:35:40.465505 31202 raft_consensus.cc:504] T 4ba16ab551a54de4862cfb1345a2af21 P 9c227810e58d448098cfb7b2f5857077 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb8a1f6b5ac48fa8a568998b3cd853f" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 44517 } } peers { permanent_uuid: "27b5a31c0cf54826abbb5628302f41f7" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 39771 } } peers { permanent_uuid: "9c227810e58d448098cfb7b2f5857077" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 42683 } }
I20260808 22:35:40.465821 31202 leader_election.cc:288] T 4ba16ab551a54de4862cfb1345a2af21 P 9c227810e58d448098cfb7b2f5857077 [CANDIDATE]: Term 1 election: Requested vote from peers 4cb8a1f6b5ac48fa8a568998b3cd853f (127.31.87.131:44517), 27b5a31c0cf54826abbb5628302f41f7 (127.31.87.130:39771)
I20260808 22:35:40.466130 30877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ba16ab551a54de4862cfb1345a2af21" candidate_uuid: "9c227810e58d448098cfb7b2f5857077" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27b5a31c0cf54826abbb5628302f41f7"
I20260808 22:35:40.466236 30877 raft_consensus.cc:3037] T 4ba16ab551a54de4862cfb1345a2af21 P 27b5a31c0cf54826abbb5628302f41f7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:35:40.467231 30877 raft_consensus.cc:2454] T 4ba16ab551a54de4862cfb1345a2af21 P 27b5a31c0cf54826abbb5628302f41f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c227810e58d448098cfb7b2f5857077 in term 1.
I20260808 22:35:40.467482 31073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ba16ab551a54de4862cfb1345a2af21" candidate_uuid: "9c227810e58d448098cfb7b2f5857077" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cb8a1f6b5ac48fa8a568998b3cd853f"
I20260808 22:35:40.467541 30584 leader_election.cc:302] T 4ba16ab551a54de4862cfb1345a2af21 P 9c227810e58d448098cfb7b2f5857077 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27b5a31c0cf54826abbb5628302f41f7, 9c227810e58d448098cfb7b2f5857077; no voters:
I20260808 22:35:40.467620 31073 raft_consensus.cc:3037] T 4ba16ab551a54de4862cfb1345a2af21 P 4cb8a1f6b5ac48fa8a568998b3cd853f [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:35:40.467689 31202 raft_consensus.cc:2781] T 4ba16ab551a54de4862cfb1345a2af21 P 9c227810e58d448098cfb7b2f5857077 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:35:40.467900 31202 raft_consensus.cc:686] T 4ba16ab551a54de4862cfb1345a2af21 P 9c227810e58d448098cfb7b2f5857077 [term 1 LEADER]: Becoming Leader. State: Replica: 9c227810e58d448098cfb7b2f5857077, State: Running, Role: LEADER
I20260808 22:35:40.468210 31202 consensus_queue.cc:237] T 4ba16ab551a54de4862cfb1345a2af21 P 9c227810e58d448098cfb7b2f5857077 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb8a1f6b5ac48fa8a568998b3cd853f" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 44517 } } peers { permanent_uuid: "27b5a31c0cf54826abbb5628302f41f7" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 39771 } } peers { permanent_uuid: "9c227810e58d448098cfb7b2f5857077" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 42683 } }
I20260808 22:35:40.468662 31073 raft_consensus.cc:2454] T 4ba16ab551a54de4862cfb1345a2af21 P 4cb8a1f6b5ac48fa8a568998b3cd853f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c227810e58d448098cfb7b2f5857077 in term 1.
I20260808 22:35:40.470969 30468 catalog_manager.cc:5495] T 4ba16ab551a54de4862cfb1345a2af21 P 9c227810e58d448098cfb7b2f5857077 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c227810e58d448098cfb7b2f5857077 (127.31.87.129). New cstate: current_term: 1 leader_uuid: "9c227810e58d448098cfb7b2f5857077" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb8a1f6b5ac48fa8a568998b3cd853f" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 44517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27b5a31c0cf54826abbb5628302f41f7" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 39771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c227810e58d448098cfb7b2f5857077" member_type: VOTER last_known_addr { host: "127.31.87.129" port: 42683 } health_report { overall_health: HEALTHY } } }
W20260808 22:35:40.478602 30751 tablet.cc:2369] T 4ba16ab551a54de4862cfb1345a2af21 P 9c227810e58d448098cfb7b2f5857077: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:35:40.489454 31187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:40.486159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33366 (local address 127.31.87.129:42683)
0808 22:35:40.486297 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:40.486301 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:40.486316 (+ 15us) server_negotiation.cc:407] Connection header received
0808 22:35:40.486448 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:40.486451 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:40.486506 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:40.486598 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:35:40.486769 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:40.487257 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:40.488200 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:40.488409 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:40.488495 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:40.488706 (+ 211us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:35:40.489125 (+ 419us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:35:40.489229 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:40.489266 (+ 37us) server_negotiation.cc:299] Negotiation successful
0808 22:35:40.489314 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 22:35:40.494776 30877 raft_consensus.cc:1261] T 4ba16ab551a54de4862cfb1345a2af21 P 27b5a31c0cf54826abbb5628302f41f7 [term 1 FOLLOWER]: Refusing update from remote peer 9c227810e58d448098cfb7b2f5857077: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:35:40.494805 31073 raft_consensus.cc:1261] T 4ba16ab551a54de4862cfb1345a2af21 P 4cb8a1f6b5ac48fa8a568998b3cd853f [term 1 FOLLOWER]: Refusing update from remote peer 9c227810e58d448098cfb7b2f5857077: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:35:40.495401 31202 consensus_queue.cc:1035] T 4ba16ab551a54de4862cfb1345a2af21 P 9c227810e58d448098cfb7b2f5857077 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27b5a31c0cf54826abbb5628302f41f7" member_type: VOTER last_known_addr { host: "127.31.87.130" port: 39771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:35:40.495590 31205 consensus_queue.cc:1035] T 4ba16ab551a54de4862cfb1345a2af21 P 9c227810e58d448098cfb7b2f5857077 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cb8a1f6b5ac48fa8a568998b3cd853f" member_type: VOTER last_known_addr { host: "127.31.87.131" port: 44517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:35:40.504834 31213 mvcc.cc:205] Tried to move back new op lower bound from 7316392101860433920 to 7316392101760339968. Current Snapshot: MvccSnapshot[applied={T|T < 7316392101860433920}]
I20260808 22:35:40.510345 30468 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49140:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:35:40.510476 30468 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49140:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:35:40.512604 30468 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a59d475cd9ac4ff3a8f5f209452bd865: Sending DeleteTablet for 3 replicas of tablet 4ba16ab551a54de4862cfb1345a2af21
I20260808 22:35:40.513343 30850 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ba16ab551a54de4862cfb1345a2af21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:35:40 UTC) from {username='slave'} at 127.0.0.1:60264
I20260808 22:35:40.513378 30651 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ba16ab551a54de4862cfb1345a2af21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:35:40 UTC) from {username='slave'} at 127.0.0.1:33362
I20260808 22:35:40.513756 31226 tablet_replica.cc:330] T 4ba16ab551a54de4862cfb1345a2af21 P 27b5a31c0cf54826abbb5628302f41f7: stopping tablet replica
I20260808 22:35:40.513926 31038 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ba16ab551a54de4862cfb1345a2af21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:35:40 UTC) from {username='slave'} at 127.0.0.1:55070
I20260808 22:35:40.514003 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 30559
I20260808 22:35:40.514045 31226 raft_consensus.cc:2229] T 4ba16ab551a54de4862cfb1345a2af21 P 27b5a31c0cf54826abbb5628302f41f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:35:40.514704 31229 tablet_replica.cc:330] T 4ba16ab551a54de4862cfb1345a2af21 P 4cb8a1f6b5ac48fa8a568998b3cd853f: stopping tablet replica
I20260808 22:35:40.514905 31229 raft_consensus.cc:2229] T 4ba16ab551a54de4862cfb1345a2af21 P 4cb8a1f6b5ac48fa8a568998b3cd853f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:35:40.515111 31229 raft_consensus.cc:2258] T 4ba16ab551a54de4862cfb1345a2af21 P 4cb8a1f6b5ac48fa8a568998b3cd853f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:35:40.519450 31226 raft_consensus.cc:2258] T 4ba16ab551a54de4862cfb1345a2af21 P 27b5a31c0cf54826abbb5628302f41f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:35:40.519763 31229 ts_tablet_manager.cc:1905] T 4ba16ab551a54de4862cfb1345a2af21 P 4cb8a1f6b5ac48fa8a568998b3cd853f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:35:40.522070 31226 ts_tablet_manager.cc:1905] T 4ba16ab551a54de4862cfb1345a2af21 P 27b5a31c0cf54826abbb5628302f41f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:35:40.523840 31229 ts_tablet_manager.cc:1918] T 4ba16ab551a54de4862cfb1345a2af21 P 4cb8a1f6b5ac48fa8a568998b3cd853f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:35:40.523940 31229 log.cc:1196] T 4ba16ab551a54de4862cfb1345a2af21 P 4cb8a1f6b5ac48fa8a568998b3cd853f: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/ts-2/wal/wals/4ba16ab551a54de4862cfb1345a2af21
I20260808 22:35:40.524150 31226 ts_tablet_manager.cc:1918] T 4ba16ab551a54de4862cfb1345a2af21 P 27b5a31c0cf54826abbb5628302f41f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:35:40.524231 31226 log.cc:1196] T 4ba16ab551a54de4862cfb1345a2af21 P 27b5a31c0cf54826abbb5628302f41f7: Deleting WAL directory at /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786228291013016-32094-0/minicluster-data/ts-1/wal/wals/4ba16ab551a54de4862cfb1345a2af21
I20260808 22:35:40.524242 31229 ts_tablet_manager.cc:1939] T 4ba16ab551a54de4862cfb1345a2af21 P 4cb8a1f6b5ac48fa8a568998b3cd853f: Deleting consensus metadata
I20260808 22:35:40.524497 31226 ts_tablet_manager.cc:1939] T 4ba16ab551a54de4862cfb1345a2af21 P 27b5a31c0cf54826abbb5628302f41f7: Deleting consensus metadata
I20260808 22:35:40.525434 30448 catalog_manager.cc:4841] TS 4cb8a1f6b5ac48fa8a568998b3cd853f (127.31.87.131:44517): tablet 4ba16ab551a54de4862cfb1345a2af21 (table test-table [id=fb94ac81aa334832a9665a0967a0047f]) successfully deleted
I20260808 22:35:40.525508 30452 catalog_manager.cc:4841] TS 27b5a31c0cf54826abbb5628302f41f7 (127.31.87.130:39771): tablet 4ba16ab551a54de4862cfb1345a2af21 (table test-table [id=fb94ac81aa334832a9665a0967a0047f]) successfully deleted
W20260808 22:35:40.528517 30448 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.87.129:42683 (error 108)
I20260808 22:35:40.528695 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 30753
I20260808 22:35:40.533780 31180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:40.533489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.129:42683 (local address 127.0.0.1:33370)
0808 22:35:40.533647 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 22:35:40.533704 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.129:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
W20260808 22:35:40.534442 30448 catalog_manager.cc:4584] TS 9c227810e58d448098cfb7b2f5857077 (127.31.87.129:42683): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4ba16ab551a54de4862cfb1345a2af21: Network error: Client connection negotiation failed: client connection to 127.31.87.129:42683: connect: Connection refused (error 111)
I20260808 22:35:40.538784 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 30944
I20260808 22:35:40.546087 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 30415
2026-08-08T22:35:40Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1016 ms)
[----------] 6 tests from EncryptionPolicyTest (10315 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786228291013016-32094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:35:40 dist-test-slave-266d krb5kdc[31234](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:35:40 dist-test-slave-266d krb5kdc[31234](info): set up 2 sockets
Aug 08 22:35:40 dist-test-slave-266d krb5kdc[31234](info): commencing operation
krb5kdc: starting...
W20260808 22:35:41.678787 32094 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Aug 08 22:35:41 dist-test-slave-266d krb5kdc[31234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228541, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:35:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:35:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786228291013016-32094-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786228291013016-32094-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 22:35:41.791869 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38113
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786228291013016-32094-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38113
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:35:41.901328 31253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:35:41.901552 31253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:35:41.901608 31253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:35:41.905470 31253 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:35:41.905521 31253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:35:41.905545 31253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:35:41.905577 31253 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:35:41.905601 31253 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:35:41.910566 31253 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786228291013016-32094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38113
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786228291013016-32094-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31253
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:35:41.911442 31253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:35:41.912336 31253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:35:41.920109 31265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:41.920195 31268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:41.920116 31264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:41.920347 31253 server_base.cc:1034] running on GCE node
I20260808 22:35:41.920928 31253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:35:41.921978 31253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:35:41.923151 31253 hybrid_clock.cc:648] HybridClock initialized: now 1786228541923137 us; error 26 us; skew 500 ppm
Aug 08 22:35:41 dist-test-slave-266d krb5kdc[31234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228541, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:35:41.925730 31253 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 22:35:41.926528 31253 webserver.cc:460] Webserver started at http://127.0.0.1:39935/ using document root <none> and password file <none>
I20260808 22:35:41.927006 31253 fs_manager.cc:359] Metadata directory not provided
I20260808 22:35:41.927062 31253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:35:41.927218 31253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:35:41.929051 31253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786228291013016-32094-0/minicluster-data/master-0/data/instance:
uuid: "c1708bd71fc445e9b4b7cb031961b332"
format_stamp: "Formatted at 2026-08-08 22:35:41 on dist-test-slave-266d"
server_key: "d3ad6b571339996fd6728e19c28d1a6f"
server_key_iv: "8d46fdcb50d614ff87d3a28eeaabc8ff"
server_key_version: "encryptionkey@0"
I20260808 22:35:41.929587 31253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786228291013016-32094-0/minicluster-data/master-0/wal/instance:
uuid: "c1708bd71fc445e9b4b7cb031961b332"
format_stamp: "Formatted at 2026-08-08 22:35:41 on dist-test-slave-266d"
server_key: "d3ad6b571339996fd6728e19c28d1a6f"
server_key_iv: "8d46fdcb50d614ff87d3a28eeaabc8ff"
server_key_version: "encryptionkey@0"
I20260808 22:35:41.932907 31253 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.002s
I20260808 22:35:41.935207 31280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:41.936331 31253 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:35:41.936429 31253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "c1708bd71fc445e9b4b7cb031961b332"
format_stamp: "Formatted at 2026-08-08 22:35:41 on dist-test-slave-266d"
server_key: "d3ad6b571339996fd6728e19c28d1a6f"
server_key_iv: "8d46fdcb50d614ff87d3a28eeaabc8ff"
server_key_version: "encryptionkey@0"
I20260808 22:35:41.936506 31253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:35:41.946209 31253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:35:41.948551 31253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:35:41.948686 31253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:35:41.955063 31253 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38113
I20260808 22:35:41.955075 31357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38113 every 8 connection(s)
I20260808 22:35:41.955904 31253 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:35:41.956552 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 31253
I20260808 22:35:41.956686 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786228291013016-32094-0/minicluster-data/master-0/wal/instance
I20260808 22:35:41.956998 32094 external_mini_cluster.cc:1428] Setting key f987417d3913b345fc58a433e8a73045
I20260808 22:35:41.958905 31359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:35:41.963963 31359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1708bd71fc445e9b4b7cb031961b332: Bootstrap starting.
Aug 08 22:35:41 dist-test-slave-266d krb5kdc[31234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228541, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 22:35:41.966323 31359 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1708bd71fc445e9b4b7cb031961b332: Neither blocks nor log segments found. Creating new log.
I20260808 22:35:41.967054 31359 log.cc:824] T 00000000000000000000000000000000 P c1708bd71fc445e9b4b7cb031961b332: Log is configured to *not* fsync() on all Append() calls
I20260808 22:35:41.968957 31359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1708bd71fc445e9b4b7cb031961b332: No bootstrap required, opened a new log
I20260808 22:35:41.970263 31362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:41.958325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52921 (local address 127.0.0.1:38113)
0808 22:35:41.958727 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:41.958741 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:41.958783 (+ 42us) server_negotiation.cc:407] Connection header received
0808 22:35:41.959397 (+ 614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:41.959414 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:41.959723 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:41.960022 (+ 299us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:41.960814 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:41.961817 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:41.962665 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:41.963114 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:41.963254 (+ 140us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:41.965907 (+ 2653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:41.965913 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:41.965924 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:35:41.965949 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:41.967620 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:35:41.968146 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:35:41.968150 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:35:41.968155 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:35:41.968270 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:35:41.968575 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:35:41.968577 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:35:41.968578 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:35:41.968916 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:41.969112 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:41.969396 (+ 284us) server_negotiation.cc:299] Negotiation successful
0808 22:35:41.969592 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":125,"threads_started":1}
I20260808 22:35:41.972612 31359 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1708bd71fc445e9b4b7cb031961b332 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1708bd71fc445e9b4b7cb031961b332" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38113 } }
I20260808 22:35:41.972855 31359 raft_consensus.cc:374] T 00000000000000000000000000000000 P c1708bd71fc445e9b4b7cb031961b332 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:35:41.972920 31359 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1708bd71fc445e9b4b7cb031961b332 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1708bd71fc445e9b4b7cb031961b332, State: Initialized, Role: FOLLOWER
I20260808 22:35:41.973002 32094 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:35:41.973372 31359 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1708bd71fc445e9b4b7cb031961b332 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1708bd71fc445e9b4b7cb031961b332" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38113 } }
I20260808 22:35:41.973501 31359 raft_consensus.cc:388] T 00000000000000000000000000000000 P c1708bd71fc445e9b4b7cb031961b332 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:35:41.973573 31359 raft_consensus.cc:482] T 00000000000000000000000000000000 P c1708bd71fc445e9b4b7cb031961b332 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:35:41.973667 31359 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1708bd71fc445e9b4b7cb031961b332 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:35:41.974484 31359 raft_consensus.cc:504] T 00000000000000000000000000000000 P c1708bd71fc445e9b4b7cb031961b332 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1708bd71fc445e9b4b7cb031961b332" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38113 } }
I20260808 22:35:41.974809 31359 leader_election.cc:302] T 00000000000000000000000000000000 P c1708bd71fc445e9b4b7cb031961b332 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c1708bd71fc445e9b4b7cb031961b332; no voters:
I20260808 22:35:41.975142 31359 leader_election.cc:288] T 00000000000000000000000000000000 P c1708bd71fc445e9b4b7cb031961b332 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:35:41.975215 31373 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c1708bd71fc445e9b4b7cb031961b332 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:35:41.975389 31373 raft_consensus.cc:686] T 00000000000000000000000000000000 P c1708bd71fc445e9b4b7cb031961b332 [term 1 LEADER]: Becoming Leader. State: Replica: c1708bd71fc445e9b4b7cb031961b332, State: Running, Role: LEADER
I20260808 22:35:41.975658 31373 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1708bd71fc445e9b4b7cb031961b332 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1708bd71fc445e9b4b7cb031961b332" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38113 } }
I20260808 22:35:41.976301 31359 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1708bd71fc445e9b4b7cb031961b332 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:35:41.976754 31373 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1708bd71fc445e9b4b7cb031961b332 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1708bd71fc445e9b4b7cb031961b332. Latest consensus state: current_term: 1 leader_uuid: "c1708bd71fc445e9b4b7cb031961b332" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1708bd71fc445e9b4b7cb031961b332" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38113 } } }
I20260808 22:35:41.976944 31373 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1708bd71fc445e9b4b7cb031961b332 [sys.catalog]: This master's current role is: LEADER
I20260808 22:35:41.977211 31373 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1708bd71fc445e9b4b7cb031961b332 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c1708bd71fc445e9b4b7cb031961b332" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1708bd71fc445e9b4b7cb031961b332" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38113 } } }
I20260808 22:35:41.977355 31373 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1708bd71fc445e9b4b7cb031961b332 [sys.catalog]: This master's current role is: LEADER
I20260808 22:35:41.977643 31383 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:35:41.981122 31383 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:35:41.983187 31362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:41.974907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50933 (local address 127.0.0.1:38113)
0808 22:35:41.975038 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:41.975042 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:41.975276 (+ 234us) server_negotiation.cc:407] Connection header received
0808 22:35:41.975395 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:41.975398 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:41.975439 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:41.975507 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:41.976230 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:41.976890 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:41.977988 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:41.978320 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:41.978398 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:41.979940 (+ 1542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:41.979944 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:41.979947 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:35:41.979971 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:41.981544 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:35:41.982028 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:35:41.982031 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:35:41.982033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:35:41.982106 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:35:41.982421 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:35:41.982424 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:35:41.982426 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:35:41.982580 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:41.982703 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:41.982971 (+ 268us) server_negotiation.cc:299] Negotiation successful
0808 22:35:41.983051 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 22:35:41.986615 31383 catalog_manager.cc:1347] Generated new cluster ID: 5fddf1e2f56e4d0d98bbde9d0caaa7d9
I20260808 22:35:41.986672 31383 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:35:42.011806 31383 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:35:42.012676 31383 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:35:42.025888 31383 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c1708bd71fc445e9b4b7cb031961b332: Generated new TSK 0
I20260808 22:35:42.026571 31383 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:35:42.032918 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 31253
W20260808 22:35:42.046005 31232 connection.cc:535] client connection to 127.0.0.1:38113 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38113): Connection reset by peer (error 104)
2026-08-08T22:35:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1489 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786228291013016-32094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:35:42 dist-test-slave-266d krb5kdc[31398](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:35:42 dist-test-slave-266d krb5kdc[31398](info): set up 2 sockets
Aug 08 22:35:42 dist-test-slave-266d krb5kdc[31398](info): commencing operation
krb5kdc: starting...
W20260808 22:35:43.164024 32094 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Aug 08 22:35:43 dist-test-slave-266d krb5kdc[31398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228543, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:35:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:35:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786228291013016-32094-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786228291013016-32094-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 22:35:43.277276 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37659
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786228291013016-32094-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37659
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:35:43.383068 31419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:35:43.383298 31419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:35:43.383337 31419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:35:43.387161 31419 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:35:43.387209 31419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:35:43.387226 31419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:35:43.387241 31419 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:35:43.387270 31419 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:35:43.392284 31419 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786228291013016-32094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37659
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786228291013016-32094-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:35:43.393216 31419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:35:43.394146 31419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:35:43.403245 31430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:43.403718 31433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:43.405341 31431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:43.405710 31419 server_base.cc:1034] running on GCE node
I20260808 22:35:43.406314 31419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:35:43.407480 31419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:35:43.408625 31419 hybrid_clock.cc:648] HybridClock initialized: now 1786228543408608 us; error 35 us; skew 500 ppm
Aug 08 22:35:43 dist-test-slave-266d krb5kdc[31398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228543, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:35:43.411232 31419 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 22:35:43.412103 31419 webserver.cc:460] Webserver started at http://127.0.0.1:35701/ using document root <none> and password file <none>
I20260808 22:35:43.412602 31419 fs_manager.cc:359] Metadata directory not provided
I20260808 22:35:43.412659 31419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:35:43.412858 31419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:35:43.414808 31419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786228291013016-32094-0/minicluster-data/master-0/data/instance:
uuid: "a5c1d66fb3414b899445cc862b45afe0"
format_stamp: "Formatted at 2026-08-08 22:35:43 on dist-test-slave-266d"
server_key: "dd4b47bb4f5c76e52a54479f0a2042cf"
server_key_iv: "65bd0d84955b2fdb19e1aa24eecc0a14"
server_key_version: "encryptionkey@0"
I20260808 22:35:43.415319 31419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786228291013016-32094-0/minicluster-data/master-0/wal/instance:
uuid: "a5c1d66fb3414b899445cc862b45afe0"
format_stamp: "Formatted at 2026-08-08 22:35:43 on dist-test-slave-266d"
server_key: "dd4b47bb4f5c76e52a54479f0a2042cf"
server_key_iv: "65bd0d84955b2fdb19e1aa24eecc0a14"
server_key_version: "encryptionkey@0"
I20260808 22:35:43.418777 31419 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 22:35:43.420924 31448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:43.422154 31419 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:35:43.422261 31419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "a5c1d66fb3414b899445cc862b45afe0"
format_stamp: "Formatted at 2026-08-08 22:35:43 on dist-test-slave-266d"
server_key: "dd4b47bb4f5c76e52a54479f0a2042cf"
server_key_iv: "65bd0d84955b2fdb19e1aa24eecc0a14"
server_key_version: "encryptionkey@0"
I20260808 22:35:43.422348 31419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:35:43.439546 31419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:35:43.442132 31419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:35:43.442307 31419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:35:43.448645 31419 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37659
I20260808 22:35:43.448689 31533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37659 every 8 connection(s)
I20260808 22:35:43.449538 31419 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:35:43.452104 31535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:35:43.452571 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 31419
I20260808 22:35:43.452698 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786228291013016-32094-0/minicluster-data/master-0/wal/instance
I20260808 22:35:43.452970 32094 external_mini_cluster.cc:1428] Setting key f7616d9165765ccf007e6db5200a68e5
I20260808 22:35:43.457923 31535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5c1d66fb3414b899445cc862b45afe0: Bootstrap starting.
I20260808 22:35:43.460304 31535 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a5c1d66fb3414b899445cc862b45afe0: Neither blocks nor log segments found. Creating new log.
I20260808 22:35:43.461048 31535 log.cc:824] T 00000000000000000000000000000000 P a5c1d66fb3414b899445cc862b45afe0: Log is configured to *not* fsync() on all Append() calls
I20260808 22:35:43.461436 31539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:43.454274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58549 (local address 127.0.0.1:37659)
0808 22:35:43.454639 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:43.454654 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:43.454687 (+ 33us) server_negotiation.cc:407] Connection header received
0808 22:35:43.455307 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:43.455322 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:43.455628 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:43.455892 (+ 264us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:43.456259 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:43.457234 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:43.458349 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:43.458798 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:43.458908 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:43.459781 (+ 873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:43.459786 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:43.459798 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:35:43.460168 (+ 370us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:43.460218 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:35:43.460241 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:43.460339 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:43.460524 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 22:35:43.460644 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":98,"threads_started":1}
I20260808 22:35:43.462982 31535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5c1d66fb3414b899445cc862b45afe0: No bootstrap required, opened a new log
I20260808 22:35:43.464140 32094 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:35:43.466655 31535 raft_consensus.cc:348] T 00000000000000000000000000000000 P a5c1d66fb3414b899445cc862b45afe0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5c1d66fb3414b899445cc862b45afe0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37659 } }
I20260808 22:35:43.466850 31535 raft_consensus.cc:374] T 00000000000000000000000000000000 P a5c1d66fb3414b899445cc862b45afe0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:35:43.466894 31535 raft_consensus.cc:729] T 00000000000000000000000000000000 P a5c1d66fb3414b899445cc862b45afe0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5c1d66fb3414b899445cc862b45afe0, State: Initialized, Role: FOLLOWER
I20260808 22:35:43.467265 31535 consensus_queue.cc:260] T 00000000000000000000000000000000 P a5c1d66fb3414b899445cc862b45afe0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5c1d66fb3414b899445cc862b45afe0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37659 } }
I20260808 22:35:43.467383 31535 raft_consensus.cc:388] T 00000000000000000000000000000000 P a5c1d66fb3414b899445cc862b45afe0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:35:43.467449 31535 raft_consensus.cc:482] T 00000000000000000000000000000000 P a5c1d66fb3414b899445cc862b45afe0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:35:43.467549 31535 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a5c1d66fb3414b899445cc862b45afe0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:35:43.468540 31535 raft_consensus.cc:504] T 00000000000000000000000000000000 P a5c1d66fb3414b899445cc862b45afe0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5c1d66fb3414b899445cc862b45afe0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37659 } }
I20260808 22:35:43.468840 31535 leader_election.cc:302] T 00000000000000000000000000000000 P a5c1d66fb3414b899445cc862b45afe0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a5c1d66fb3414b899445cc862b45afe0; no voters:
I20260808 22:35:43.469144 31535 leader_election.cc:288] T 00000000000000000000000000000000 P a5c1d66fb3414b899445cc862b45afe0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:35:43.469215 31548 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a5c1d66fb3414b899445cc862b45afe0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:35:43.469412 31548 raft_consensus.cc:686] T 00000000000000000000000000000000 P a5c1d66fb3414b899445cc862b45afe0 [term 1 LEADER]: Becoming Leader. State: Replica: a5c1d66fb3414b899445cc862b45afe0, State: Running, Role: LEADER
I20260808 22:35:43.469719 31548 consensus_queue.cc:237] T 00000000000000000000000000000000 P a5c1d66fb3414b899445cc862b45afe0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5c1d66fb3414b899445cc862b45afe0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37659 } }
I20260808 22:35:43.470247 31539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:43.466000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50565 (local address 127.0.0.1:37659)
0808 22:35:43.466093 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:43.466097 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:43.466253 (+ 156us) server_negotiation.cc:407] Connection header received
0808 22:35:43.466372 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:43.466375 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:43.466412 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:43.466471 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:43.466714 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:43.467326 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:43.468400 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:43.468646 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:43.468742 (+ 96us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:43.469696 (+ 954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:43.469699 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:43.469701 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:35:43.469715 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:43.469742 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:35:43.469755 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:43.469842 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:43.470103 (+ 261us) server_negotiation.cc:299] Negotiation successful
0808 22:35:43.470134 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 22:35:43.470386 31535 sys_catalog.cc:564] T 00000000000000000000000000000000 P a5c1d66fb3414b899445cc862b45afe0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:35:43.470825 31548 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5c1d66fb3414b899445cc862b45afe0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a5c1d66fb3414b899445cc862b45afe0. Latest consensus state: current_term: 1 leader_uuid: "a5c1d66fb3414b899445cc862b45afe0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5c1d66fb3414b899445cc862b45afe0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37659 } } }
I20260808 22:35:43.470929 31548 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5c1d66fb3414b899445cc862b45afe0 [sys.catalog]: This master's current role is: LEADER
I20260808 22:35:43.471269 31560 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:35:43.471913 31549 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5c1d66fb3414b899445cc862b45afe0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a5c1d66fb3414b899445cc862b45afe0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5c1d66fb3414b899445cc862b45afe0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37659 } } }
I20260808 22:35:43.472075 31549 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5c1d66fb3414b899445cc862b45afe0 [sys.catalog]: This master's current role is: LEADER
I20260808 22:35:43.474640 31560 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:35:43.479851 31560 catalog_manager.cc:1347] Generated new cluster ID: 54651fe8622c4a02838e6da1e5f10492
I20260808 22:35:43.479909 31560 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:35:43.486914 31560 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:35:43.487655 31560 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:35:43.496945 31560 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a5c1d66fb3414b899445cc862b45afe0: Generated new TSK 0
I20260808 22:35:43.497509 31560 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:35:43.504194 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 31419
W20260808 22:35:43.519052 31396 connection.cc:535] client connection to 127.0.0.1:37659 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37659): Connection reset by peer (error 104)
2026-08-08T22:35:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1472 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786228291013016-32094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:35:43 dist-test-slave-266d krb5kdc[31583](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:35:43 dist-test-slave-266d krb5kdc[31583](info): set up 2 sockets
Aug 08 22:35:43 dist-test-slave-266d krb5kdc[31583](info): commencing operation
krb5kdc: starting...
W20260808 22:35:44.653576 32094 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Aug 08 22:35:44 dist-test-slave-266d krb5kdc[31583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228544, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:35:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:35:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786228291013016-32094-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786228291013016-32094-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 22:35:44.770026 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44025
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786228291013016-32094-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44025
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:35:44.876859 31604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:35:44.877117 31604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:35:44.877164 31604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:35:44.881004 31604 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:35:44.881053 31604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:35:44.881071 31604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:35:44.881103 31604 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:35:44.881124 31604 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:35:44.886127 31604 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786228291013016-32094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44025
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786228291013016-32094-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44025
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31604
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:35:44.887046 31604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:35:44.887962 31604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:35:44.895951 31609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:44.896031 31610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:44.896080 31604 server_base.cc:1034] running on GCE node
W20260808 22:35:44.895951 31613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:44.896721 31604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:35:44.897753 31604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:35:44.898916 31604 hybrid_clock.cc:648] HybridClock initialized: now 1786228544898900 us; error 26 us; skew 500 ppm
Aug 08 22:35:44 dist-test-slave-266d krb5kdc[31583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228544, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:35:44.901369 31604 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 22:35:44.902191 31604 webserver.cc:460] Webserver started at http://127.0.0.1:39527/ using document root <none> and password file <none>
I20260808 22:35:44.902691 31604 fs_manager.cc:359] Metadata directory not provided
I20260808 22:35:44.902753 31604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:35:44.902961 31604 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:35:44.904853 31604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786228291013016-32094-0/minicluster-data/master-0/data/instance:
uuid: "dd9ef44ac30b4dcf9b5a1e45d3e16aee"
format_stamp: "Formatted at 2026-08-08 22:35:44 on dist-test-slave-266d"
server_key: "8ff3b1147f2319f1d18f5839e7a4ac2f"
server_key_iv: "e191d1e6dd7fb57bffe9d03290ae804e"
server_key_version: "encryptionkey@0"
I20260808 22:35:44.905385 31604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786228291013016-32094-0/minicluster-data/master-0/wal/instance:
uuid: "dd9ef44ac30b4dcf9b5a1e45d3e16aee"
format_stamp: "Formatted at 2026-08-08 22:35:44 on dist-test-slave-266d"
server_key: "8ff3b1147f2319f1d18f5839e7a4ac2f"
server_key_iv: "e191d1e6dd7fb57bffe9d03290ae804e"
server_key_version: "encryptionkey@0"
I20260808 22:35:44.908738 31604 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.001s
I20260808 22:35:44.910938 31629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:44.912052 31604 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:35:44.912158 31604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "dd9ef44ac30b4dcf9b5a1e45d3e16aee"
format_stamp: "Formatted at 2026-08-08 22:35:44 on dist-test-slave-266d"
server_key: "8ff3b1147f2319f1d18f5839e7a4ac2f"
server_key_iv: "e191d1e6dd7fb57bffe9d03290ae804e"
server_key_version: "encryptionkey@0"
I20260808 22:35:44.912245 31604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:35:44.916524 31604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:35:44.919044 31604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:35:44.919194 31604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:35:44.925710 31604 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44025
I20260808 22:35:44.925730 31712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44025 every 8 connection(s)
I20260808 22:35:44.926527 31604 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:35:44.929046 31713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:35:44.934226 31713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd9ef44ac30b4dcf9b5a1e45d3e16aee: Bootstrap starting.
I20260808 22:35:44.936501 31713 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd9ef44ac30b4dcf9b5a1e45d3e16aee: Neither blocks nor log segments found. Creating new log.
I20260808 22:35:44.936582 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 31604
I20260808 22:35:44.936739 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786228291013016-32094-0/minicluster-data/master-0/wal/instance
I20260808 22:35:44.937048 32094 external_mini_cluster.cc:1428] Setting key a5d99b3e550933dbfba57213cd8e8605
I20260808 22:35:44.937381 31713 log.cc:824] T 00000000000000000000000000000000 P dd9ef44ac30b4dcf9b5a1e45d3e16aee: Log is configured to *not* fsync() on all Append() calls
I20260808 22:35:44.939271 31713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd9ef44ac30b4dcf9b5a1e45d3e16aee: No bootstrap required, opened a new log
I20260808 22:35:44.941807 31721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:44.938045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37025 (local address 127.0.0.1:44025)
0808 22:35:44.938449 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:44.938462 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:44.938494 (+ 32us) server_negotiation.cc:407] Connection header received
0808 22:35:44.939097 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:44.939114 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:44.939362 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:44.939674 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:44.940405 (+ 731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:44.940411 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:44.940421 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:35:44.940431 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:44.940483 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:35:44.940502 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:44.940614 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:44.940754 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 22:35:44.940913 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":114,"threads_started":1}
I20260808 22:35:44.942343 31713 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd9ef44ac30b4dcf9b5a1e45d3e16aee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd9ef44ac30b4dcf9b5a1e45d3e16aee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44025 } }
I20260808 22:35:44.942587 31713 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd9ef44ac30b4dcf9b5a1e45d3e16aee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:35:44.942654 31713 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd9ef44ac30b4dcf9b5a1e45d3e16aee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd9ef44ac30b4dcf9b5a1e45d3e16aee, State: Initialized, Role: FOLLOWER
I20260808 22:35:44.943048 31713 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd9ef44ac30b4dcf9b5a1e45d3e16aee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd9ef44ac30b4dcf9b5a1e45d3e16aee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44025 } }
I20260808 22:35:44.943181 31713 raft_consensus.cc:388] T 00000000000000000000000000000000 P dd9ef44ac30b4dcf9b5a1e45d3e16aee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:35:44.943243 31713 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd9ef44ac30b4dcf9b5a1e45d3e16aee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:35:44.943339 31713 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd9ef44ac30b4dcf9b5a1e45d3e16aee [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:35:44.944276 31713 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd9ef44ac30b4dcf9b5a1e45d3e16aee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd9ef44ac30b4dcf9b5a1e45d3e16aee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44025 } }
I20260808 22:35:44.944351 32094 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:35:44.944588 31713 leader_election.cc:302] T 00000000000000000000000000000000 P dd9ef44ac30b4dcf9b5a1e45d3e16aee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd9ef44ac30b4dcf9b5a1e45d3e16aee; no voters:
I20260808 22:35:44.944859 31713 leader_election.cc:288] T 00000000000000000000000000000000 P dd9ef44ac30b4dcf9b5a1e45d3e16aee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:35:44.944933 31723 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd9ef44ac30b4dcf9b5a1e45d3e16aee [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:35:44.945133 31723 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd9ef44ac30b4dcf9b5a1e45d3e16aee [term 1 LEADER]: Becoming Leader. State: Replica: dd9ef44ac30b4dcf9b5a1e45d3e16aee, State: Running, Role: LEADER
I20260808 22:35:44.945467 31723 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd9ef44ac30b4dcf9b5a1e45d3e16aee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd9ef44ac30b4dcf9b5a1e45d3e16aee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44025 } }
I20260808 22:35:44.946069 31713 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd9ef44ac30b4dcf9b5a1e45d3e16aee [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:35:44.947388 31724 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd9ef44ac30b4dcf9b5a1e45d3e16aee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dd9ef44ac30b4dcf9b5a1e45d3e16aee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd9ef44ac30b4dcf9b5a1e45d3e16aee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44025 } } }
I20260808 22:35:44.947523 31724 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd9ef44ac30b4dcf9b5a1e45d3e16aee [sys.catalog]: This master's current role is: LEADER
I20260808 22:35:44.947734 31724 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd9ef44ac30b4dcf9b5a1e45d3e16aee [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd9ef44ac30b4dcf9b5a1e45d3e16aee. Latest consensus state: current_term: 1 leader_uuid: "dd9ef44ac30b4dcf9b5a1e45d3e16aee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd9ef44ac30b4dcf9b5a1e45d3e16aee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44025 } } }
I20260808 22:35:44.947813 31724 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd9ef44ac30b4dcf9b5a1e45d3e16aee [sys.catalog]: This master's current role is: LEADER
I20260808 22:35:44.947888 31750 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:35:44.948072 31721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:44.946603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55033 (local address 127.0.0.1:44025)
0808 22:35:44.946694 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:44.946698 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:44.946709 (+ 11us) server_negotiation.cc:407] Connection header received
0808 22:35:44.946739 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:44.946742 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:44.946769 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:44.946836 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:44.947457 (+ 621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:44.947460 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:44.947463 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:35:44.947473 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:44.947498 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:35:44.947510 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:44.947580 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:44.947936 (+ 356us) server_negotiation.cc:299] Negotiation successful
0808 22:35:44.947974 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 22:35:44.951084 31750 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:35:44.956195 31750 catalog_manager.cc:1347] Generated new cluster ID: 526b4d84080742fab029ea772413b796
I20260808 22:35:44.956260 31750 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:35:44.972947 31750 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:35:44.973271 31750 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:35:44.986500 31750 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dd9ef44ac30b4dcf9b5a1e45d3e16aee: Generated new TSK 0
I20260808 22:35:44.987142 31750 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:35:44.991518 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 31604
2026-08-08T22:35:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1492 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786228291013016-32094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:35:45 dist-test-slave-266d krb5kdc[31769](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:35:45 dist-test-slave-266d krb5kdc[31769](info): set up 2 sockets
Aug 08 22:35:45 dist-test-slave-266d krb5kdc[31769](info): commencing operation
krb5kdc: starting...
W20260808 22:35:46.135746 32094 mini_kdc.cc:121] Time spent starting KDC: real 1.110s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Aug 08 22:35:46 dist-test-slave-266d krb5kdc[31769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228546, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:35:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:35:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786228291013016-32094-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786228291013016-32094-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 22:35:46.249287 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34429
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786228291013016-32094-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34429
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:35:46.354661 31793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:35:46.354910 31793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:35:46.354966 31793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:35:46.358866 31793 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:35:46.358916 31793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:35:46.358943 31793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:35:46.358965 31793 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:35:46.358994 31793 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:35:46.363932 31793 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786228291013016-32094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34429
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786228291013016-32094-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31793
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:35:46.364863 31793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:35:46.365821 31793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:35:46.373747 31801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:46.373745 31806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:46.373962 31793 server_base.cc:1034] running on GCE node
W20260808 22:35:46.373867 31802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:46.374524 31793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:35:46.375533 31793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:35:46.376693 31793 hybrid_clock.cc:648] HybridClock initialized: now 1786228546376679 us; error 25 us; skew 500 ppm
Aug 08 22:35:46 dist-test-slave-266d krb5kdc[31769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228546, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:35:46.381992 31793 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 22:35:46.382792 31793 webserver.cc:460] Webserver started at http://127.0.0.1:35699/ using document root <none> and password file <none>
I20260808 22:35:46.383284 31793 fs_manager.cc:359] Metadata directory not provided
I20260808 22:35:46.383342 31793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:35:46.383502 31793 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:35:46.385424 31793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786228291013016-32094-0/minicluster-data/master-0/data/instance:
uuid: "6916f4c68c3b43ac928f836cb8acd054"
format_stamp: "Formatted at 2026-08-08 22:35:46 on dist-test-slave-266d"
server_key: "46f30657416a4318da3bdd856312f15c"
server_key_iv: "fe21a4f064495790eaf89467da803554"
server_key_version: "encryptionkey@0"
I20260808 22:35:46.385921 31793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786228291013016-32094-0/minicluster-data/master-0/wal/instance:
uuid: "6916f4c68c3b43ac928f836cb8acd054"
format_stamp: "Formatted at 2026-08-08 22:35:46 on dist-test-slave-266d"
server_key: "46f30657416a4318da3bdd856312f15c"
server_key_iv: "fe21a4f064495790eaf89467da803554"
server_key_version: "encryptionkey@0"
I20260808 22:35:46.389299 31793 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 22:35:46.391489 31816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:46.392584 31793 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:35:46.392696 31793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "6916f4c68c3b43ac928f836cb8acd054"
format_stamp: "Formatted at 2026-08-08 22:35:46 on dist-test-slave-266d"
server_key: "46f30657416a4318da3bdd856312f15c"
server_key_iv: "fe21a4f064495790eaf89467da803554"
server_key_version: "encryptionkey@0"
I20260808 22:35:46.392781 31793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:35:46.420455 31793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:35:46.423221 31793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:35:46.423410 31793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:35:46.430403 31793 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34429
I20260808 22:35:46.430410 31898 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34429 every 8 connection(s)
I20260808 22:35:46.431324 31793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:35:46.433888 31899 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:35:46.434582 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 31793
I20260808 22:35:46.434719 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786228291013016-32094-0/minicluster-data/master-0/wal/instance
I20260808 22:35:46.435038 32094 external_mini_cluster.cc:1428] Setting key 6cd92c7d6b406932f011f7af4938db76
I20260808 22:35:46.440362 31899 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6916f4c68c3b43ac928f836cb8acd054: Bootstrap starting.
I20260808 22:35:46.442677 31899 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6916f4c68c3b43ac928f836cb8acd054: Neither blocks nor log segments found. Creating new log.
I20260808 22:35:46.443329 31899 log.cc:824] T 00000000000000000000000000000000 P 6916f4c68c3b43ac928f836cb8acd054: Log is configured to *not* fsync() on all Append() calls
Aug 08 22:35:46 dist-test-slave-266d krb5kdc[31769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228546, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 22:35:46.445278 31899 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6916f4c68c3b43ac928f836cb8acd054: No bootstrap required, opened a new log
I20260808 22:35:46.448479 31899 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6916f4c68c3b43ac928f836cb8acd054 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6916f4c68c3b43ac928f836cb8acd054" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34429 } }
I20260808 22:35:46.448673 31899 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6916f4c68c3b43ac928f836cb8acd054 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:35:46.448678 31904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:46.436337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38663 (local address 127.0.0.1:34429)
0808 22:35:46.436759 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:46.436777 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:46.436818 (+ 41us) server_negotiation.cc:407] Connection header received
0808 22:35:46.437637 (+ 819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:46.437660 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:46.438055 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:46.438395 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:46.439054 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:46.440495 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:46.441357 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:46.442002 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:46.442151 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:46.444531 (+ 2380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:46.444537 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:46.444551 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:35:46.444580 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:46.446165 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:35:46.446562 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:35:46.446568 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:35:46.446574 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:35:46.446684 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:35:46.446932 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:35:46.446936 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:35:46.446937 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:35:46.447299 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:46.447478 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:46.447740 (+ 262us) server_negotiation.cc:299] Negotiation successful
0808 22:35:46.448003 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":110,"threads_started":1}
I20260808 22:35:46.448729 31899 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6916f4c68c3b43ac928f836cb8acd054 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6916f4c68c3b43ac928f836cb8acd054, State: Initialized, Role: FOLLOWER
I20260808 22:35:46.449182 31899 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6916f4c68c3b43ac928f836cb8acd054 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6916f4c68c3b43ac928f836cb8acd054" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34429 } }
I20260808 22:35:46.449313 31899 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6916f4c68c3b43ac928f836cb8acd054 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:35:46.449378 31899 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6916f4c68c3b43ac928f836cb8acd054 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:35:46.449468 31899 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6916f4c68c3b43ac928f836cb8acd054 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:35:46.450425 31899 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6916f4c68c3b43ac928f836cb8acd054 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6916f4c68c3b43ac928f836cb8acd054" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34429 } }
I20260808 22:35:46.450759 31899 leader_election.cc:302] T 00000000000000000000000000000000 P 6916f4c68c3b43ac928f836cb8acd054 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6916f4c68c3b43ac928f836cb8acd054; no voters:
I20260808 22:35:46.451081 31899 leader_election.cc:288] T 00000000000000000000000000000000 P 6916f4c68c3b43ac928f836cb8acd054 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:35:46.451166 31906 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6916f4c68c3b43ac928f836cb8acd054 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:35:46.451383 31906 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6916f4c68c3b43ac928f836cb8acd054 [term 1 LEADER]: Becoming Leader. State: Replica: 6916f4c68c3b43ac928f836cb8acd054, State: Running, Role: LEADER
I20260808 22:35:46.451548 32094 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:35:46.451691 31906 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6916f4c68c3b43ac928f836cb8acd054 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6916f4c68c3b43ac928f836cb8acd054" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34429 } }
I20260808 22:35:46.452229 31899 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6916f4c68c3b43ac928f836cb8acd054 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:35:46.453146 31909 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6916f4c68c3b43ac928f836cb8acd054 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6916f4c68c3b43ac928f836cb8acd054" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6916f4c68c3b43ac928f836cb8acd054" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34429 } } }
I20260808 22:35:46.453266 31909 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6916f4c68c3b43ac928f836cb8acd054 [sys.catalog]: This master's current role is: LEADER
I20260808 22:35:46.453574 31912 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6916f4c68c3b43ac928f836cb8acd054 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6916f4c68c3b43ac928f836cb8acd054. Latest consensus state: current_term: 1 leader_uuid: "6916f4c68c3b43ac928f836cb8acd054" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6916f4c68c3b43ac928f836cb8acd054" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34429 } } }
I20260808 22:35:46.453691 31912 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6916f4c68c3b43ac928f836cb8acd054 [sys.catalog]: This master's current role is: LEADER
I20260808 22:35:46.454011 31922 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:35:46.457239 31922 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:35:46.462630 31904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:46.453574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52487 (local address 127.0.0.1:34429)
0808 22:35:46.453668 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:46.453672 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:46.454148 (+ 476us) server_negotiation.cc:407] Connection header received
0808 22:35:46.454328 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:46.454331 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:46.454372 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:46.454418 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:46.455204 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:46.455922 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:46.457024 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:46.457366 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:46.457468 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:46.458976 (+ 1508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:46.458981 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:46.458983 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:35:46.459008 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:46.460612 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:35:46.461078 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:35:46.461081 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:35:46.461083 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:35:46.461187 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:35:46.461772 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:35:46.461776 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:35:46.461778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:35:46.461991 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:46.462138 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:46.462377 (+ 239us) server_negotiation.cc:299] Negotiation successful
0808 22:35:46.462490 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 22:35:46.462925 31922 catalog_manager.cc:1347] Generated new cluster ID: 62064166e026457697d22c524acefbf3
I20260808 22:35:46.463009 31922 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:35:46.475044 31922 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:35:46.475835 31922 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:35:46.483104 31922 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6916f4c68c3b43ac928f836cb8acd054: Generated new TSK 0
I20260808 22:35:46.483655 31922 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:35:46.490936 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 31793
2026-08-08T22:35:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1487 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786228291013016-32094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:35:46 dist-test-slave-266d krb5kdc[31954](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:35:46 dist-test-slave-266d krb5kdc[31954](info): set up 2 sockets
Aug 08 22:35:46 dist-test-slave-266d krb5kdc[31954](info): commencing operation
krb5kdc: starting...
W20260808 22:35:47.624393 32094 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Aug 08 22:35:47 dist-test-slave-266d krb5kdc[31954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228547, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:35:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:35:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786228291013016-32094-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786228291013016-32094-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 22:35:47.741995 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43549
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786228291013016-32094-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43549
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:35:47.850016 31978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:35:47.850252 31978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:35:47.850291 31978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:35:47.854158 31978 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:35:47.854208 31978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:35:47.854226 31978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:35:47.854246 31978 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:35:47.854260 31978 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:35:47.859380 31978 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786228291013016-32094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43549
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786228291013016-32094-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43549
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31978
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:35:47.860322 31978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:35:47.861311 31978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:35:47.869167 31986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:47.869199 31987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:47.869287 31978 server_base.cc:1034] running on GCE node
W20260808 22:35:47.869177 31991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:47.869957 31978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:35:47.870970 31978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:35:47.872140 31978 hybrid_clock.cc:648] HybridClock initialized: now 1786228547872127 us; error 25 us; skew 500 ppm
Aug 08 22:35:47 dist-test-slave-266d krb5kdc[31954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228547, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:35:47.874646 31978 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 22:35:47.875442 31978 webserver.cc:460] Webserver started at http://127.0.0.1:37349/ using document root <none> and password file <none>
I20260808 22:35:47.875943 31978 fs_manager.cc:359] Metadata directory not provided
I20260808 22:35:47.876006 31978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:35:47.876212 31978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:35:47.878177 31978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786228291013016-32094-0/minicluster-data/master-0/data/instance:
uuid: "96769632f88f4a3eb780c4bc5e50c2e4"
format_stamp: "Formatted at 2026-08-08 22:35:47 on dist-test-slave-266d"
server_key: "1a8a9c8d001cb596da2e33d2345fac3f"
server_key_iv: "097f57d4f3097be88667799b0d2e7ecb"
server_key_version: "encryptionkey@0"
I20260808 22:35:47.878695 31978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786228291013016-32094-0/minicluster-data/master-0/wal/instance:
uuid: "96769632f88f4a3eb780c4bc5e50c2e4"
format_stamp: "Formatted at 2026-08-08 22:35:47 on dist-test-slave-266d"
server_key: "1a8a9c8d001cb596da2e33d2345fac3f"
server_key_iv: "097f57d4f3097be88667799b0d2e7ecb"
server_key_version: "encryptionkey@0"
I20260808 22:35:47.882143 31978 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260808 22:35:47.884290 31999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:47.885529 31978 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 22:35:47.885671 31978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "96769632f88f4a3eb780c4bc5e50c2e4"
format_stamp: "Formatted at 2026-08-08 22:35:47 on dist-test-slave-266d"
server_key: "1a8a9c8d001cb596da2e33d2345fac3f"
server_key_iv: "097f57d4f3097be88667799b0d2e7ecb"
server_key_version: "encryptionkey@0"
I20260808 22:35:47.885769 31978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:35:47.914674 31978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:35:47.917375 31978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:35:47.917567 31978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:35:47.924536 31978 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43549
I20260808 22:35:47.924547 32081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43549 every 8 connection(s)
I20260808 22:35:47.925493 31978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:35:47.927394 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 31978
I20260808 22:35:47.927551 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786228291013016-32094-0/minicluster-data/master-0/wal/instance
I20260808 22:35:47.927850 32094 external_mini_cluster.cc:1428] Setting key 30a0b6a72a369fbcf00419f81e758615
I20260808 22:35:47.928330 32082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:35:47.934513 32082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96769632f88f4a3eb780c4bc5e50c2e4: Bootstrap starting.
I20260808 22:35:47.936345 32087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:47.929166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57423 (local address 127.0.0.1:43549)
0808 22:35:47.929566 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:47.929587 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:47.929624 (+ 37us) server_negotiation.cc:407] Connection header received
0808 22:35:47.930260 (+ 636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:47.930276 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:47.930593 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:47.930861 (+ 268us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:47.931162 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:47.932125 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:47.933323 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:47.933786 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:47.933908 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:47.934728 (+ 820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:47.934734 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:47.934745 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:35:47.935124 (+ 379us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:47.935180 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:35:47.935206 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:47.935287 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:47.935440 (+ 153us) server_negotiation.cc:299] Negotiation successful
0808 22:35:47.935625 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":125,"threads_started":1}
I20260808 22:35:47.936671 32082 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 96769632f88f4a3eb780c4bc5e50c2e4: Neither blocks nor log segments found. Creating new log.
I20260808 22:35:47.937422 32082 log.cc:824] T 00000000000000000000000000000000 P 96769632f88f4a3eb780c4bc5e50c2e4: Log is configured to *not* fsync() on all Append() calls
I20260808 22:35:47.939093 32094 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:35:47.939395 32082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96769632f88f4a3eb780c4bc5e50c2e4: No bootstrap required, opened a new log
I20260808 22:35:47.943521 32082 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96769632f88f4a3eb780c4bc5e50c2e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96769632f88f4a3eb780c4bc5e50c2e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43549 } }
I20260808 22:35:47.943739 32082 raft_consensus.cc:374] T 00000000000000000000000000000000 P 96769632f88f4a3eb780c4bc5e50c2e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:35:47.943799 32082 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96769632f88f4a3eb780c4bc5e50c2e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96769632f88f4a3eb780c4bc5e50c2e4, State: Initialized, Role: FOLLOWER
I20260808 22:35:47.944269 32082 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96769632f88f4a3eb780c4bc5e50c2e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96769632f88f4a3eb780c4bc5e50c2e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43549 } }
I20260808 22:35:47.944432 32082 raft_consensus.cc:388] T 00000000000000000000000000000000 P 96769632f88f4a3eb780c4bc5e50c2e4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:35:47.944501 32082 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96769632f88f4a3eb780c4bc5e50c2e4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:35:47.944610 32082 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96769632f88f4a3eb780c4bc5e50c2e4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:35:47.945024 32087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:47.941212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34683 (local address 127.0.0.1:43549)
0808 22:35:47.941330 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:47.941335 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:47.941489 (+ 154us) server_negotiation.cc:407] Connection header received
0808 22:35:47.941594 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:47.941596 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:47.941634 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:47.941702 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:47.941963 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:47.942430 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:47.943492 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:47.943740 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:47.943848 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:47.944599 (+ 751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:47.944605 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:47.944608 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:35:47.944623 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:47.944649 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:35:47.944663 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:47.944746 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:47.944843 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 22:35:47.944892 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 22:35:47.945592 32082 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96769632f88f4a3eb780c4bc5e50c2e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96769632f88f4a3eb780c4bc5e50c2e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43549 } }
I20260808 22:35:47.945916 32082 leader_election.cc:302] T 00000000000000000000000000000000 P 96769632f88f4a3eb780c4bc5e50c2e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 96769632f88f4a3eb780c4bc5e50c2e4; no voters:
I20260808 22:35:47.946211 32082 leader_election.cc:288] T 00000000000000000000000000000000 P 96769632f88f4a3eb780c4bc5e50c2e4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:35:47.946322 32108 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96769632f88f4a3eb780c4bc5e50c2e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:35:47.946539 32108 raft_consensus.cc:686] T 00000000000000000000000000000000 P 96769632f88f4a3eb780c4bc5e50c2e4 [term 1 LEADER]: Becoming Leader. State: Replica: 96769632f88f4a3eb780c4bc5e50c2e4, State: Running, Role: LEADER
I20260808 22:35:47.946898 32108 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96769632f88f4a3eb780c4bc5e50c2e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96769632f88f4a3eb780c4bc5e50c2e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43549 } }
I20260808 22:35:47.947368 32082 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96769632f88f4a3eb780c4bc5e50c2e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:35:47.948594 32111 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96769632f88f4a3eb780c4bc5e50c2e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96769632f88f4a3eb780c4bc5e50c2e4. Latest consensus state: current_term: 1 leader_uuid: "96769632f88f4a3eb780c4bc5e50c2e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96769632f88f4a3eb780c4bc5e50c2e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43549 } } }
I20260808 22:35:47.948722 32111 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96769632f88f4a3eb780c4bc5e50c2e4 [sys.catalog]: This master's current role is: LEADER
I20260808 22:35:47.949110 32123 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:35:47.949038 32109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96769632f88f4a3eb780c4bc5e50c2e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "96769632f88f4a3eb780c4bc5e50c2e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96769632f88f4a3eb780c4bc5e50c2e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43549 } } }
I20260808 22:35:47.949180 32109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96769632f88f4a3eb780c4bc5e50c2e4 [sys.catalog]: This master's current role is: LEADER
I20260808 22:35:47.951727 32123 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:35:47.956338 32123 catalog_manager.cc:1347] Generated new cluster ID: eee29ada3b35470ab123f759ba249f7b
I20260808 22:35:47.956398 32123 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:35:47.965638 32123 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:35:47.966426 32123 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:35:47.977741 32123 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 96769632f88f4a3eb780c4bc5e50c2e4: Generated new TSK 0
I20260808 22:35:47.978327 32123 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:35:47.986073 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 31978
2026-08-08T22:35:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1497 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786228291013016-32094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:35:48 dist-test-slave-266d krb5kdc[32136](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:35:48 dist-test-slave-266d krb5kdc[32136](info): set up 2 sockets
Aug 08 22:35:48 dist-test-slave-266d krb5kdc[32136](info): commencing operation
krb5kdc: starting...
W20260808 22:35:49.128623 32094 mini_kdc.cc:121] Time spent starting KDC: real 1.118s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Aug 08 22:35:49 dist-test-slave-266d krb5kdc[32136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:35:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:35:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786228291013016-32094-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786228291013016-32094-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 22:35:49.242376 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44873
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786228291013016-32094-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44873
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:35:49.348781 32157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:35:49.349028 32157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:35:49.349067 32157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:35:49.352996 32157 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:35:49.353058 32157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:35:49.353076 32157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:35:49.353111 32157 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:35:49.353133 32157 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:35:49.358342 32157 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786228291013016-32094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44873
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786228291013016-32094-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32157
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:35:49.359303 32157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:35:49.360232 32157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:35:49.369655 32164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:49.369660 32165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:49.369992 32169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:49.370163 32157 server_base.cc:1034] running on GCE node
I20260808 22:35:49.370977 32157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:35:49.372344 32157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:35:49.373518 32157 hybrid_clock.cc:648] HybridClock initialized: now 1786228549373490 us; error 43 us; skew 500 ppm
Aug 08 22:35:49 dist-test-slave-266d krb5kdc[32136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228549, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:35:49.376856 32157 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 22:35:49.377794 32157 webserver.cc:460] Webserver started at http://127.0.0.1:35345/ using document root <none> and password file <none>
I20260808 22:35:49.378336 32157 fs_manager.cc:359] Metadata directory not provided
I20260808 22:35:49.378458 32157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:35:49.378708 32157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:35:49.380879 32157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786228291013016-32094-0/minicluster-data/master-0/data/instance:
uuid: "eaebab71e91a474c9e48768ce5f94789"
format_stamp: "Formatted at 2026-08-08 22:35:49 on dist-test-slave-266d"
server_key: "4291c30fb441ab4e4ca308dedd0ce3a8"
server_key_iv: "6e1d7a66c20662a6aecb300951ff2548"
server_key_version: "encryptionkey@0"
I20260808 22:35:49.381529 32157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786228291013016-32094-0/minicluster-data/master-0/wal/instance:
uuid: "eaebab71e91a474c9e48768ce5f94789"
format_stamp: "Formatted at 2026-08-08 22:35:49 on dist-test-slave-266d"
server_key: "4291c30fb441ab4e4ca308dedd0ce3a8"
server_key_iv: "6e1d7a66c20662a6aecb300951ff2548"
server_key_version: "encryptionkey@0"
I20260808 22:35:49.386329 32157 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 22:35:49.389513 32180 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:49.392338 32157 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260808 22:35:49.392469 32157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "eaebab71e91a474c9e48768ce5f94789"
format_stamp: "Formatted at 2026-08-08 22:35:49 on dist-test-slave-266d"
server_key: "4291c30fb441ab4e4ca308dedd0ce3a8"
server_key_iv: "6e1d7a66c20662a6aecb300951ff2548"
server_key_version: "encryptionkey@0"
I20260808 22:35:49.392621 32157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:35:49.398775 32157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:35:49.402788 32157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:35:49.402990 32157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:35:49.412063 32157 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44873
I20260808 22:35:49.412065 32260 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44873 every 8 connection(s)
I20260808 22:35:49.413275 32157 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:35:49.416005 32261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:35:49.417961 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 32157
I20260808 22:35:49.418099 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786228291013016-32094-0/minicluster-data/master-0/wal/instance
I20260808 22:35:49.418381 32094 external_mini_cluster.cc:1428] Setting key 68bbe9259e6b8164668922f4f726c982
I20260808 22:35:49.422734 32261 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eaebab71e91a474c9e48768ce5f94789: Bootstrap starting.
I20260808 22:35:49.423079 32265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:49.419591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33611 (local address 127.0.0.1:44873)
0808 22:35:49.419950 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:49.419965 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:49.419999 (+ 34us) server_negotiation.cc:407] Connection header received
0808 22:35:49.420613 (+ 614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:49.420638 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:49.420901 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:49.421222 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:49.421925 (+ 703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:49.421929 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:49.421941 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:35:49.421950 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:49.421996 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:35:49.422013 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:49.422117 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:49.422227 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 22:35:49.422370 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":97,"threads_started":1}
I20260808 22:35:49.425047 32261 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eaebab71e91a474c9e48768ce5f94789: Neither blocks nor log segments found. Creating new log.
I20260808 22:35:49.425697 32094 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:35:49.425803 32261 log.cc:824] T 00000000000000000000000000000000 P eaebab71e91a474c9e48768ce5f94789: Log is configured to *not* fsync() on all Append() calls
I20260808 22:35:49.427605 32261 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eaebab71e91a474c9e48768ce5f94789: No bootstrap required, opened a new log
I20260808 22:35:49.430452 32265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:49.427703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51593 (local address 127.0.0.1:44873)
0808 22:35:49.427806 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:49.427810 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:49.429358 (+ 1548us) server_negotiation.cc:407] Connection header received
0808 22:35:49.429465 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:49.429466 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:49.429494 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:49.429553 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:49.430142 (+ 589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:49.430144 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:49.430146 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:35:49.430156 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:49.430177 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:35:49.430187 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:49.430247 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:49.430339 (+ 92us) server_negotiation.cc:299] Negotiation successful
0808 22:35:49.430366 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 22:35:49.431334 32261 raft_consensus.cc:348] T 00000000000000000000000000000000 P eaebab71e91a474c9e48768ce5f94789 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaebab71e91a474c9e48768ce5f94789" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44873 } }
I20260808 22:35:49.431532 32261 raft_consensus.cc:374] T 00000000000000000000000000000000 P eaebab71e91a474c9e48768ce5f94789 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:35:49.431588 32261 raft_consensus.cc:729] T 00000000000000000000000000000000 P eaebab71e91a474c9e48768ce5f94789 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaebab71e91a474c9e48768ce5f94789, State: Initialized, Role: FOLLOWER
I20260808 22:35:49.431979 32261 consensus_queue.cc:260] T 00000000000000000000000000000000 P eaebab71e91a474c9e48768ce5f94789 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaebab71e91a474c9e48768ce5f94789" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44873 } }
I20260808 22:35:49.432103 32261 raft_consensus.cc:388] T 00000000000000000000000000000000 P eaebab71e91a474c9e48768ce5f94789 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:35:49.432166 32261 raft_consensus.cc:482] T 00000000000000000000000000000000 P eaebab71e91a474c9e48768ce5f94789 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:35:49.432257 32261 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eaebab71e91a474c9e48768ce5f94789 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:35:49.433002 32261 raft_consensus.cc:504] T 00000000000000000000000000000000 P eaebab71e91a474c9e48768ce5f94789 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaebab71e91a474c9e48768ce5f94789" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44873 } }
I20260808 22:35:49.433326 32261 leader_election.cc:302] T 00000000000000000000000000000000 P eaebab71e91a474c9e48768ce5f94789 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eaebab71e91a474c9e48768ce5f94789; no voters:
I20260808 22:35:49.433596 32261 leader_election.cc:288] T 00000000000000000000000000000000 P eaebab71e91a474c9e48768ce5f94789 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:35:49.433683 32274 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eaebab71e91a474c9e48768ce5f94789 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:35:49.433871 32274 raft_consensus.cc:686] T 00000000000000000000000000000000 P eaebab71e91a474c9e48768ce5f94789 [term 1 LEADER]: Becoming Leader. State: Replica: eaebab71e91a474c9e48768ce5f94789, State: Running, Role: LEADER
I20260808 22:35:49.434221 32274 consensus_queue.cc:237] T 00000000000000000000000000000000 P eaebab71e91a474c9e48768ce5f94789 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaebab71e91a474c9e48768ce5f94789" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44873 } }
I20260808 22:35:49.434670 32261 sys_catalog.cc:564] T 00000000000000000000000000000000 P eaebab71e91a474c9e48768ce5f94789 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:35:49.435783 32279 sys_catalog.cc:455] T 00000000000000000000000000000000 P eaebab71e91a474c9e48768ce5f94789 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eaebab71e91a474c9e48768ce5f94789. Latest consensus state: current_term: 1 leader_uuid: "eaebab71e91a474c9e48768ce5f94789" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaebab71e91a474c9e48768ce5f94789" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44873 } } }
I20260808 22:35:49.435897 32279 sys_catalog.cc:458] T 00000000000000000000000000000000 P eaebab71e91a474c9e48768ce5f94789 [sys.catalog]: This master's current role is: LEADER
I20260808 22:35:49.436000 32276 sys_catalog.cc:455] T 00000000000000000000000000000000 P eaebab71e91a474c9e48768ce5f94789 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eaebab71e91a474c9e48768ce5f94789" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaebab71e91a474c9e48768ce5f94789" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44873 } } }
I20260808 22:35:49.436102 32276 sys_catalog.cc:458] T 00000000000000000000000000000000 P eaebab71e91a474c9e48768ce5f94789 [sys.catalog]: This master's current role is: LEADER
I20260808 22:35:49.436379 32288 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:35:49.438925 32288 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:35:49.444141 32288 catalog_manager.cc:1347] Generated new cluster ID: 76242a89dc5b46888968557fa41078c4
I20260808 22:35:49.444205 32288 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:35:49.476866 32288 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:35:49.477180 32288 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:35:49.486682 32288 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eaebab71e91a474c9e48768ce5f94789: Generated new TSK 0
I20260808 22:35:49.487257 32288 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:35:49.495335 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 32157
2026-08-08T22:35:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1513 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786228291013016-32094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:35:49 dist-test-slave-266d krb5kdc[32306](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:35:49 dist-test-slave-266d krb5kdc[32306](info): set up 2 sockets
Aug 08 22:35:49 dist-test-slave-266d krb5kdc[32306](info): commencing operation
krb5kdc: starting...
W20260808 22:35:50.643187 32094 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Aug 08 22:35:50 dist-test-slave-266d krb5kdc[32306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228550, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:35:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:35:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786228291013016-32094-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786228291013016-32094-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 22:35:50.757819 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34529
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786228291013016-32094-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34529
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:35:50.865415 32325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:35:50.865634 32325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:35:50.865681 32325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:35:50.869503 32325 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:35:50.869549 32325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:35:50.869567 32325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:35:50.869588 32325 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:35:50.869602 32325 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:35:50.874534 32325 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786228291013016-32094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34529
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786228291013016-32094-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34529
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32325
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:35:50.875394 32325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:35:50.876278 32325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:35:50.884372 32340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:50.884508 32325 server_base.cc:1034] running on GCE node
W20260808 22:35:50.884413 32336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:50.884428 32332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:50.885151 32325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:35:50.886185 32325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:35:50.887346 32325 hybrid_clock.cc:648] HybridClock initialized: now 1786228550887331 us; error 28 us; skew 500 ppm
Aug 08 22:35:50 dist-test-slave-266d krb5kdc[32306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228550, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:35:50.889886 32325 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 22:35:50.890741 32325 webserver.cc:460] Webserver started at http://127.0.0.1:44267/ using document root <none> and password file <none>
I20260808 22:35:50.891266 32325 fs_manager.cc:359] Metadata directory not provided
I20260808 22:35:50.891332 32325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:35:50.891538 32325 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:35:50.893548 32325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786228291013016-32094-0/minicluster-data/master-0/data/instance:
uuid: "f22828c123b94177a12a46f023804f6d"
format_stamp: "Formatted at 2026-08-08 22:35:50 on dist-test-slave-266d"
server_key: "8288eb3d600fbc2c07da7ca335253144"
server_key_iv: "3023c0fca4515f4d85b7554703feda26"
server_key_version: "encryptionkey@0"
I20260808 22:35:50.894068 32325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786228291013016-32094-0/minicluster-data/master-0/wal/instance:
uuid: "f22828c123b94177a12a46f023804f6d"
format_stamp: "Formatted at 2026-08-08 22:35:50 on dist-test-slave-266d"
server_key: "8288eb3d600fbc2c07da7ca335253144"
server_key_iv: "3023c0fca4515f4d85b7554703feda26"
server_key_version: "encryptionkey@0"
I20260808 22:35:50.897631 32325 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 22:35:50.899832 32350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:50.900952 32325 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:35:50.901065 32325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "f22828c123b94177a12a46f023804f6d"
format_stamp: "Formatted at 2026-08-08 22:35:50 on dist-test-slave-266d"
server_key: "8288eb3d600fbc2c07da7ca335253144"
server_key_iv: "3023c0fca4515f4d85b7554703feda26"
server_key_version: "encryptionkey@0"
I20260808 22:35:50.901194 32325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:35:50.914498 32325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:35:50.917023 32325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:35:50.917222 32325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:35:50.923864 32325 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34529
I20260808 22:35:50.923873 32435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34529 every 8 connection(s)
I20260808 22:35:50.924768 32325 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:35:50.927343 32436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:35:50.932578 32436 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f22828c123b94177a12a46f023804f6d: Bootstrap starting.
I20260808 22:35:50.933238 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 32325
I20260808 22:35:50.933372 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786228291013016-32094-0/minicluster-data/master-0/wal/instance
I20260808 22:35:50.933632 32094 external_mini_cluster.cc:1428] Setting key a8a2c1174a2596062df056891f0f1b6e
I20260808 22:35:50.935218 32436 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f22828c123b94177a12a46f023804f6d: Neither blocks nor log segments found. Creating new log.
I20260808 22:35:50.935892 32436 log.cc:824] T 00000000000000000000000000000000 P f22828c123b94177a12a46f023804f6d: Log is configured to *not* fsync() on all Append() calls
I20260808 22:35:50.937978 32436 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f22828c123b94177a12a46f023804f6d: No bootstrap required, opened a new log
I20260808 22:35:50.941933 32436 raft_consensus.cc:348] T 00000000000000000000000000000000 P f22828c123b94177a12a46f023804f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f22828c123b94177a12a46f023804f6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34529 } }
I20260808 22:35:50.942198 32436 raft_consensus.cc:374] T 00000000000000000000000000000000 P f22828c123b94177a12a46f023804f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:35:50.942260 32436 raft_consensus.cc:729] T 00000000000000000000000000000000 P f22828c123b94177a12a46f023804f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f22828c123b94177a12a46f023804f6d, State: Initialized, Role: FOLLOWER
Aug 08 22:35:50 dist-test-slave-266d krb5kdc[32306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228550, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 22:35:50.942651 32436 consensus_queue.cc:260] T 00000000000000000000000000000000 P f22828c123b94177a12a46f023804f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f22828c123b94177a12a46f023804f6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34529 } }
I20260808 22:35:50.942797 32436 raft_consensus.cc:388] T 00000000000000000000000000000000 P f22828c123b94177a12a46f023804f6d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:35:50.942854 32436 raft_consensus.cc:482] T 00000000000000000000000000000000 P f22828c123b94177a12a46f023804f6d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:35:50.942947 32436 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f22828c123b94177a12a46f023804f6d [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:35:50.943732 32436 raft_consensus.cc:504] T 00000000000000000000000000000000 P f22828c123b94177a12a46f023804f6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f22828c123b94177a12a46f023804f6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34529 } }
I20260808 22:35:50.944041 32436 leader_election.cc:302] T 00000000000000000000000000000000 P f22828c123b94177a12a46f023804f6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f22828c123b94177a12a46f023804f6d; no voters:
I20260808 22:35:50.944296 32436 leader_election.cc:288] T 00000000000000000000000000000000 P f22828c123b94177a12a46f023804f6d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:35:50.944435 32446 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f22828c123b94177a12a46f023804f6d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:35:50.944646 32446 raft_consensus.cc:686] T 00000000000000000000000000000000 P f22828c123b94177a12a46f023804f6d [term 1 LEADER]: Becoming Leader. State: Replica: f22828c123b94177a12a46f023804f6d, State: Running, Role: LEADER
I20260808 22:35:50.945013 32446 consensus_queue.cc:237] T 00000000000000000000000000000000 P f22828c123b94177a12a46f023804f6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f22828c123b94177a12a46f023804f6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34529 } }
I20260808 22:35:50.945448 32436 sys_catalog.cc:564] T 00000000000000000000000000000000 P f22828c123b94177a12a46f023804f6d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:35:50.946305 32451 sys_catalog.cc:455] T 00000000000000000000000000000000 P f22828c123b94177a12a46f023804f6d [sys.catalog]: SysCatalogTable state changed. Reason: New leader f22828c123b94177a12a46f023804f6d. Latest consensus state: current_term: 1 leader_uuid: "f22828c123b94177a12a46f023804f6d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f22828c123b94177a12a46f023804f6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34529 } } }
I20260808 22:35:50.946430 32451 sys_catalog.cc:458] T 00000000000000000000000000000000 P f22828c123b94177a12a46f023804f6d [sys.catalog]: This master's current role is: LEADER
I20260808 22:35:50.946760 32450 sys_catalog.cc:455] T 00000000000000000000000000000000 P f22828c123b94177a12a46f023804f6d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f22828c123b94177a12a46f023804f6d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f22828c123b94177a12a46f023804f6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34529 } } }
I20260808 22:35:50.946846 32450 sys_catalog.cc:458] T 00000000000000000000000000000000 P f22828c123b94177a12a46f023804f6d [sys.catalog]: This master's current role is: LEADER
I20260808 22:35:50.947088 32463 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:35:50.949203 32440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:50.934896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.165:49329 (local address 127.0.0.1:34529)
0808 22:35:50.935333 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:50.935354 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:50.935391 (+ 37us) server_negotiation.cc:407] Connection header received
0808 22:35:50.936003 (+ 612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:50.936026 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:50.936449 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:50.936771 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:50.937640 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:50.938622 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:50.939856 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:50.940332 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:50.940455 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:50.943318 (+ 2863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:50.943323 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:50.943336 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:35:50.943363 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:50.945323 (+ 1960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:35:50.945888 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:35:50.945893 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:35:50.945899 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:35:50.946036 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:35:50.946460 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:35:50.946464 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:35:50.946466 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:35:50.946952 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:50.947185 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:50.948067 (+ 882us) server_negotiation.cc:299] Negotiation successful
0808 22:35:50.948293 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":129,"threads_started":1}
I20260808 22:35:50.950842 32463 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:35:50.952598 32094 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:35:50.957007 32463 catalog_manager.cc:1347] Generated new cluster ID: 1fe5c22c1d3c4bc1a8ce0af085295e4d
I20260808 22:35:50.957068 32463 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:35:50.962320 32440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:50.954720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.165:40647 (local address 127.0.0.1:34529)
0808 22:35:50.954820 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:50.954824 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:50.955084 (+ 260us) server_negotiation.cc:407] Connection header received
0808 22:35:50.955235 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:50.955238 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:50.955320 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:50.955431 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:50.956172 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:50.956910 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:50.957763 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:50.958011 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:50.958096 (+ 85us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:50.959329 (+ 1233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:50.959332 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:50.959334 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:35:50.959354 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:50.960673 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:35:50.961127 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:35:50.961130 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:35:50.961132 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:35:50.961202 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:35:50.961516 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:35:50.961519 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:35:50.961521 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:35:50.961674 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:50.961808 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:50.962090 (+ 282us) server_negotiation.cc:299] Negotiation successful
0808 22:35:50.962179 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 22:35:50.976538 32463 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:35:50.977451 32463 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:35:50.984843 32463 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f22828c123b94177a12a46f023804f6d: Generated new TSK 0
I20260808 22:35:50.985502 32463 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:35:50.995864 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 32325
W20260808 22:35:51.022241 32303 connection.cc:535] client connection to 127.0.0.1:34529 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34529): Connection reset by peer (error 104)
2026-08-08T22:35:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1512 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786228291013016-32094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:35:51 dist-test-slave-266d krb5kdc[32493](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:35:51 dist-test-slave-266d krb5kdc[32493](info): set up 2 sockets
Aug 08 22:35:51 dist-test-slave-266d krb5kdc[32493](info): commencing operation
krb5kdc: starting...
W20260808 22:35:52.151021 32094 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Aug 08 22:35:52 dist-test-slave-266d krb5kdc[32493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228552, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:35:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:35:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786228291013016-32094-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786228291013016-32094-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 22:35:52.263722 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37543
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786228291013016-32094-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37543
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:35:52.372747 32509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:35:52.372998 32509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:35:52.373050 32509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:35:52.376914 32509 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:35:52.376971 32509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:35:52.376998 32509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:35:52.377036 32509 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:35:52.377069 32509 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:35:52.382165 32509 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786228291013016-32094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37543
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786228291013016-32094-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37543
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:35:52.383087 32509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:35:52.384023 32509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:35:52.392025 32520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:52.392010 32517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:52.392086 32518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:52.392406 32509 server_base.cc:1034] running on GCE node
I20260808 22:35:52.392966 32509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:35:52.394021 32509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:35:52.395187 32509 hybrid_clock.cc:648] HybridClock initialized: now 1786228552395163 us; error 34 us; skew 500 ppm
Aug 08 22:35:52 dist-test-slave-266d krb5kdc[32493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228552, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:35:52.400981 32509 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 22:35:52.401911 32509 webserver.cc:460] Webserver started at http://127.0.0.1:39705/ using document root <none> and password file <none>
I20260808 22:35:52.402439 32509 fs_manager.cc:359] Metadata directory not provided
I20260808 22:35:52.402513 32509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:35:52.402748 32509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:35:52.404736 32509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786228291013016-32094-0/minicluster-data/master-0/data/instance:
uuid: "db5290885a4b47e49c0cd3abd07ac173"
format_stamp: "Formatted at 2026-08-08 22:35:52 on dist-test-slave-266d"
server_key: "cdc10db34e1b3637b1808607812ddbb6"
server_key_iv: "3f8733dc76e017ac52c0b210f23cc59a"
server_key_version: "encryptionkey@0"
I20260808 22:35:52.405332 32509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786228291013016-32094-0/minicluster-data/master-0/wal/instance:
uuid: "db5290885a4b47e49c0cd3abd07ac173"
format_stamp: "Formatted at 2026-08-08 22:35:52 on dist-test-slave-266d"
server_key: "cdc10db34e1b3637b1808607812ddbb6"
server_key_iv: "3f8733dc76e017ac52c0b210f23cc59a"
server_key_version: "encryptionkey@0"
I20260808 22:35:52.408927 32509 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 22:35:52.411262 32527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:52.412365 32509 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:35:52.412485 32509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "db5290885a4b47e49c0cd3abd07ac173"
format_stamp: "Formatted at 2026-08-08 22:35:52 on dist-test-slave-266d"
server_key: "cdc10db34e1b3637b1808607812ddbb6"
server_key_iv: "3f8733dc76e017ac52c0b210f23cc59a"
server_key_version: "encryptionkey@0"
I20260808 22:35:52.412606 32509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:35:52.447068 32509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:35:52.450078 32509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:35:52.450273 32509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:35:52.457453 32509 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37543
I20260808 22:35:52.457468 32608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37543 every 8 connection(s)
I20260808 22:35:52.458381 32509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:35:52.460172 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 32509
I20260808 22:35:52.460309 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786228291013016-32094-0/minicluster-data/master-0/wal/instance
I20260808 22:35:52.460597 32094 external_mini_cluster.cc:1428] Setting key e7eb279964311c1d9baaac2dab07f19c
I20260808 22:35:52.461213 32610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:35:52.467442 32610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db5290885a4b47e49c0cd3abd07ac173: Bootstrap starting.
I20260808 22:35:52.469142 32614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:52.461977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.165:54751 (local address 127.0.0.1:37543)
0808 22:35:52.462354 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:52.462369 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:52.462401 (+ 32us) server_negotiation.cc:407] Connection header received
0808 22:35:52.463069 (+ 668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:52.463087 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:52.463406 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:52.463661 (+ 255us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:52.464013 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:52.464962 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:52.466164 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:52.466600 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:52.466706 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:52.467544 (+ 838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:52.467552 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:52.467568 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:35:52.467933 (+ 365us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:52.467983 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:35:52.468006 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:52.468102 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:52.468291 (+ 189us) server_negotiation.cc:299] Negotiation successful
0808 22:35:52.468410 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":103,"threads_started":1}
I20260808 22:35:52.469712 32610 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db5290885a4b47e49c0cd3abd07ac173: Neither blocks nor log segments found. Creating new log.
I20260808 22:35:52.470508 32610 log.cc:824] T 00000000000000000000000000000000 P db5290885a4b47e49c0cd3abd07ac173: Log is configured to *not* fsync() on all Append() calls
I20260808 22:35:52.471830 32094 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:35:52.472438 32610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db5290885a4b47e49c0cd3abd07ac173: No bootstrap required, opened a new log
I20260808 22:35:52.476559 32610 raft_consensus.cc:348] T 00000000000000000000000000000000 P db5290885a4b47e49c0cd3abd07ac173 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db5290885a4b47e49c0cd3abd07ac173" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37543 } }
I20260808 22:35:52.476788 32610 raft_consensus.cc:374] T 00000000000000000000000000000000 P db5290885a4b47e49c0cd3abd07ac173 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:35:52.476843 32610 raft_consensus.cc:729] T 00000000000000000000000000000000 P db5290885a4b47e49c0cd3abd07ac173 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db5290885a4b47e49c0cd3abd07ac173, State: Initialized, Role: FOLLOWER
I20260808 22:35:52.477267 32610 consensus_queue.cc:260] T 00000000000000000000000000000000 P db5290885a4b47e49c0cd3abd07ac173 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db5290885a4b47e49c0cd3abd07ac173" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37543 } }
I20260808 22:35:52.477403 32610 raft_consensus.cc:388] T 00000000000000000000000000000000 P db5290885a4b47e49c0cd3abd07ac173 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:35:52.477453 32614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:52.473674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.165:36977 (local address 127.0.0.1:37543)
0808 22:35:52.473774 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:52.473778 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:52.473905 (+ 127us) server_negotiation.cc:407] Connection header received
0808 22:35:52.473994 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:52.473997 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:52.474039 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:52.474127 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:52.474318 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:52.475059 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:52.475805 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:52.476121 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:52.476225 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:52.477007 (+ 782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:52.477010 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:52.477013 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:35:52.477026 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:52.477054 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:35:52.477068 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:52.477173 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:52.477295 (+ 122us) server_negotiation.cc:299] Negotiation successful
0808 22:35:52.477326 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 22:35:52.477473 32610 raft_consensus.cc:482] T 00000000000000000000000000000000 P db5290885a4b47e49c0cd3abd07ac173 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:35:52.477603 32610 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db5290885a4b47e49c0cd3abd07ac173 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:35:52.478524 32610 raft_consensus.cc:504] T 00000000000000000000000000000000 P db5290885a4b47e49c0cd3abd07ac173 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db5290885a4b47e49c0cd3abd07ac173" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37543 } }
I20260808 22:35:52.478842 32610 leader_election.cc:302] T 00000000000000000000000000000000 P db5290885a4b47e49c0cd3abd07ac173 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db5290885a4b47e49c0cd3abd07ac173; no voters:
I20260808 22:35:52.479105 32610 leader_election.cc:288] T 00000000000000000000000000000000 P db5290885a4b47e49c0cd3abd07ac173 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:35:52.479204 32625 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db5290885a4b47e49c0cd3abd07ac173 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:35:52.479410 32625 raft_consensus.cc:686] T 00000000000000000000000000000000 P db5290885a4b47e49c0cd3abd07ac173 [term 1 LEADER]: Becoming Leader. State: Replica: db5290885a4b47e49c0cd3abd07ac173, State: Running, Role: LEADER
I20260808 22:35:52.479681 32625 consensus_queue.cc:237] T 00000000000000000000000000000000 P db5290885a4b47e49c0cd3abd07ac173 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db5290885a4b47e49c0cd3abd07ac173" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37543 } }
I20260808 22:35:52.480198 32610 sys_catalog.cc:564] T 00000000000000000000000000000000 P db5290885a4b47e49c0cd3abd07ac173 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:35:52.480679 32625 sys_catalog.cc:455] T 00000000000000000000000000000000 P db5290885a4b47e49c0cd3abd07ac173 [sys.catalog]: SysCatalogTable state changed. Reason: New leader db5290885a4b47e49c0cd3abd07ac173. Latest consensus state: current_term: 1 leader_uuid: "db5290885a4b47e49c0cd3abd07ac173" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db5290885a4b47e49c0cd3abd07ac173" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37543 } } }
I20260808 22:35:52.480823 32625 sys_catalog.cc:458] T 00000000000000000000000000000000 P db5290885a4b47e49c0cd3abd07ac173 [sys.catalog]: This master's current role is: LEADER
I20260808 22:35:52.481174 32629 sys_catalog.cc:455] T 00000000000000000000000000000000 P db5290885a4b47e49c0cd3abd07ac173 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "db5290885a4b47e49c0cd3abd07ac173" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db5290885a4b47e49c0cd3abd07ac173" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37543 } } }
I20260808 22:35:52.481293 32639 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:35:52.481352 32629 sys_catalog.cc:458] T 00000000000000000000000000000000 P db5290885a4b47e49c0cd3abd07ac173 [sys.catalog]: This master's current role is: LEADER
I20260808 22:35:52.484138 32639 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:35:52.489511 32639 catalog_manager.cc:1347] Generated new cluster ID: 6589c0c3fdf245dfbaf4849330e6f3df
I20260808 22:35:52.489593 32639 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:35:52.498672 32639 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:35:52.499862 32639 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:35:52.514361 32639 catalog_manager.cc:5868] T 00000000000000000000000000000000 P db5290885a4b47e49c0cd3abd07ac173: Generated new TSK 0
I20260808 22:35:52.515048 32639 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:35:52.521459 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 32509
W20260808 22:35:52.532824 32491 connection.cc:535] client connection to 127.0.0.1:37543 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37543): Connection reset by peer (error 104)
2026-08-08T22:35:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1510 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786228291013016-32094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:35:52 dist-test-slave-266d krb5kdc[32663](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:35:52 dist-test-slave-266d krb5kdc[32663](info): set up 2 sockets
Aug 08 22:35:52 dist-test-slave-266d krb5kdc[32663](info): commencing operation
krb5kdc: starting...
W20260808 22:35:53.681340 32094 mini_kdc.cc:121] Time spent starting KDC: real 1.134s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Aug 08 22:35:53 dist-test-slave-266d krb5kdc[32663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228553, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:35:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:35:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786228291013016-32094-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786228291013016-32094-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 22:35:53.794247 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46013
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786228291013016-32094-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46013
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:35:53.901190 32681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:35:53.901412 32681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:35:53.901450 32681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:35:53.905400 32681 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:35:53.905452 32681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:35:53.905470 32681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:35:53.905490 32681 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:35:53.905505 32681 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:35:53.910468 32681 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786228291013016-32094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46013
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786228291013016-32094-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:35:53.911365 32681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:35:53.912250 32681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:35:53.920185 32692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:53.920186 32688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:53.920228 32689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:53.920297 32681 server_base.cc:1034] running on GCE node
I20260808 22:35:53.920910 32681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:35:53.921981 32681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:35:53.923144 32681 hybrid_clock.cc:648] HybridClock initialized: now 1786228553923128 us; error 26 us; skew 500 ppm
Aug 08 22:35:53 dist-test-slave-266d krb5kdc[32663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228553, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:35:53.925673 32681 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 22:35:53.926471 32681 webserver.cc:460] Webserver started at http://127.0.0.1:33247/ using document root <none> and password file <none>
I20260808 22:35:53.926967 32681 fs_manager.cc:359] Metadata directory not provided
I20260808 22:35:53.927031 32681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:35:53.927237 32681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:35:53.929147 32681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786228291013016-32094-0/minicluster-data/master-0/data/instance:
uuid: "d067e36a89a74bf4ad37d7536f1325be"
format_stamp: "Formatted at 2026-08-08 22:35:53 on dist-test-slave-266d"
server_key: "a3844469784ba6e29d15e5a453b29a09"
server_key_iv: "6623cfb6c3b78c44011f76bd193c82d5"
server_key_version: "encryptionkey@0"
I20260808 22:35:53.929664 32681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786228291013016-32094-0/minicluster-data/master-0/wal/instance:
uuid: "d067e36a89a74bf4ad37d7536f1325be"
format_stamp: "Formatted at 2026-08-08 22:35:53 on dist-test-slave-266d"
server_key: "a3844469784ba6e29d15e5a453b29a09"
server_key_iv: "6623cfb6c3b78c44011f76bd193c82d5"
server_key_version: "encryptionkey@0"
I20260808 22:35:53.933000 32681 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 22:35:53.935158 32706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:53.936260 32681 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:35:53.936390 32681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "d067e36a89a74bf4ad37d7536f1325be"
format_stamp: "Formatted at 2026-08-08 22:35:53 on dist-test-slave-266d"
server_key: "a3844469784ba6e29d15e5a453b29a09"
server_key_iv: "6623cfb6c3b78c44011f76bd193c82d5"
server_key_version: "encryptionkey@0"
I20260808 22:35:53.936477 32681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:35:53.940676 32681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:35:53.943482 32681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:35:53.943640 32681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:35:53.949952 32681 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46013
I20260808 22:35:53.949990 314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46013 every 8 connection(s)
I20260808 22:35:53.950773 32681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:35:53.953301 315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:35:53.958278 315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d067e36a89a74bf4ad37d7536f1325be: Bootstrap starting.
I20260808 22:35:53.959148 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 32681
I20260808 22:35:53.959275 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786228291013016-32094-0/minicluster-data/master-0/wal/instance
I20260808 22:35:53.959524 32094 external_mini_cluster.cc:1428] Setting key 89ae6e4352618cc8b73fcf8e7998b023
I20260808 22:35:53.960814 315 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d067e36a89a74bf4ad37d7536f1325be: Neither blocks nor log segments found. Creating new log.
I20260808 22:35:53.961719 315 log.cc:824] T 00000000000000000000000000000000 P d067e36a89a74bf4ad37d7536f1325be: Log is configured to *not* fsync() on all Append() calls
I20260808 22:35:53.963778 315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d067e36a89a74bf4ad37d7536f1325be: No bootstrap required, opened a new log
I20260808 22:35:53.964501 321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:53.960504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.165:47063 (local address 127.0.0.1:46013)
0808 22:35:53.960991 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:53.961010 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:53.961052 (+ 42us) server_negotiation.cc:407] Connection header received
0808 22:35:53.961856 (+ 804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:53.961879 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:53.962203 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:53.962511 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:53.963214 (+ 703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:53.963220 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:53.963234 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:35:53.963246 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:53.963313 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:35:53.963340 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:53.963440 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:53.963530 (+ 90us) server_negotiation.cc:299] Negotiation successful
0808 22:35:53.963672 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":125,"threads_started":1}
I20260808 22:35:53.967168 32094 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:35:53.967592 315 raft_consensus.cc:348] T 00000000000000000000000000000000 P d067e36a89a74bf4ad37d7536f1325be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d067e36a89a74bf4ad37d7536f1325be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46013 } }
I20260808 22:35:53.967787 315 raft_consensus.cc:374] T 00000000000000000000000000000000 P d067e36a89a74bf4ad37d7536f1325be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:35:53.967846 315 raft_consensus.cc:729] T 00000000000000000000000000000000 P d067e36a89a74bf4ad37d7536f1325be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d067e36a89a74bf4ad37d7536f1325be, State: Initialized, Role: FOLLOWER
I20260808 22:35:53.968230 315 consensus_queue.cc:260] T 00000000000000000000000000000000 P d067e36a89a74bf4ad37d7536f1325be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d067e36a89a74bf4ad37d7536f1325be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46013 } }
I20260808 22:35:53.968411 315 raft_consensus.cc:388] T 00000000000000000000000000000000 P d067e36a89a74bf4ad37d7536f1325be [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:35:53.968487 315 raft_consensus.cc:482] T 00000000000000000000000000000000 P d067e36a89a74bf4ad37d7536f1325be [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:35:53.968586 315 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d067e36a89a74bf4ad37d7536f1325be [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:35:53.969516 315 raft_consensus.cc:504] T 00000000000000000000000000000000 P d067e36a89a74bf4ad37d7536f1325be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d067e36a89a74bf4ad37d7536f1325be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46013 } }
I20260808 22:35:53.969888 315 leader_election.cc:302] T 00000000000000000000000000000000 P d067e36a89a74bf4ad37d7536f1325be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d067e36a89a74bf4ad37d7536f1325be; no voters:
I20260808 22:35:53.970216 315 leader_election.cc:288] T 00000000000000000000000000000000 P d067e36a89a74bf4ad37d7536f1325be [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:35:53.970299 329 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d067e36a89a74bf4ad37d7536f1325be [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:35:53.970526 329 raft_consensus.cc:686] T 00000000000000000000000000000000 P d067e36a89a74bf4ad37d7536f1325be [term 1 LEADER]: Becoming Leader. State: Replica: d067e36a89a74bf4ad37d7536f1325be, State: Running, Role: LEADER
I20260808 22:35:53.970815 321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:53.969189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.165:33063 (local address 127.0.0.1:46013)
0808 22:35:53.969312 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:53.969317 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:53.969493 (+ 176us) server_negotiation.cc:407] Connection header received
0808 22:35:53.969612 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:53.969615 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:53.969652 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:53.969730 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:53.970397 (+ 667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:53.970400 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:53.970403 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:35:53.970416 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:53.970442 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:35:53.970455 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:53.970510 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:53.970673 (+ 163us) server_negotiation.cc:299] Negotiation successful
0808 22:35:53.970710 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 22:35:53.970894 329 consensus_queue.cc:237] T 00000000000000000000000000000000 P d067e36a89a74bf4ad37d7536f1325be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d067e36a89a74bf4ad37d7536f1325be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46013 } }
I20260808 22:35:53.971452 315 sys_catalog.cc:564] T 00000000000000000000000000000000 P d067e36a89a74bf4ad37d7536f1325be [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:35:53.972601 332 sys_catalog.cc:455] T 00000000000000000000000000000000 P d067e36a89a74bf4ad37d7536f1325be [sys.catalog]: SysCatalogTable state changed. Reason: New leader d067e36a89a74bf4ad37d7536f1325be. Latest consensus state: current_term: 1 leader_uuid: "d067e36a89a74bf4ad37d7536f1325be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d067e36a89a74bf4ad37d7536f1325be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46013 } } }
I20260808 22:35:53.972712 332 sys_catalog.cc:458] T 00000000000000000000000000000000 P d067e36a89a74bf4ad37d7536f1325be [sys.catalog]: This master's current role is: LEADER
I20260808 22:35:53.972996 331 sys_catalog.cc:455] T 00000000000000000000000000000000 P d067e36a89a74bf4ad37d7536f1325be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d067e36a89a74bf4ad37d7536f1325be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d067e36a89a74bf4ad37d7536f1325be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46013 } } }
I20260808 22:35:53.973115 349 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:35:53.973130 331 sys_catalog.cc:458] T 00000000000000000000000000000000 P d067e36a89a74bf4ad37d7536f1325be [sys.catalog]: This master's current role is: LEADER
I20260808 22:35:53.976383 349 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:35:53.981510 349 catalog_manager.cc:1347] Generated new cluster ID: d0f0785dd22149c6a0f87f8c14177187
I20260808 22:35:53.981573 349 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:35:54.003949 349 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:35:54.004271 349 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:35:54.008919 349 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d067e36a89a74bf4ad37d7536f1325be: Generated new TSK 0
I20260808 22:35:54.009603 349 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:35:54.016013 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 32681
2026-08-08T22:35:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1500 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786228291013016-32094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:35:54 dist-test-slave-266d krb5kdc[364](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:35:54 dist-test-slave-266d krb5kdc[364](info): set up 2 sockets
Aug 08 22:35:54 dist-test-slave-266d krb5kdc[364](info): commencing operation
krb5kdc: starting...
W20260808 22:35:55.175762 32094 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Aug 08 22:35:55 dist-test-slave-266d krb5kdc[364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228555, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:35:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:35:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786228291013016-32094-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786228291013016-32094-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 22:35:55.289884 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38267
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786228291013016-32094-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38267
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:35:55.395578 382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:35:55.395802 382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:35:55.395841 382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:35:55.399667 382 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:35:55.399713 382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:35:55.399731 382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:35:55.399752 382 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:35:55.399766 382 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:35:55.404645 382 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786228291013016-32094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38267
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786228291013016-32094-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.382
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:35:55.405505 382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:35:55.406361 382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:35:55.414188 388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:55.414196 396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:55.414249 400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:55.414366 382 server_base.cc:1034] running on GCE node
I20260808 22:35:55.414945 382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:35:55.415910 382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:35:55.417140 382 hybrid_clock.cc:648] HybridClock initialized: now 1786228555417119 us; error 27 us; skew 500 ppm
Aug 08 22:35:55 dist-test-slave-266d krb5kdc[364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228555, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:35:55.419566 382 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 22:35:55.420348 382 webserver.cc:460] Webserver started at http://127.0.0.1:37085/ using document root <none> and password file <none>
I20260808 22:35:55.420845 382 fs_manager.cc:359] Metadata directory not provided
I20260808 22:35:55.421036 382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:35:55.421306 382 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:35:55.423254 382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786228291013016-32094-0/minicluster-data/master-0/data/instance:
uuid: "9332021a27a947c2aae7db392e9b2910"
format_stamp: "Formatted at 2026-08-08 22:35:55 on dist-test-slave-266d"
server_key: "83287fb7f65c79fe301fc8dfaba267a1"
server_key_iv: "911cf64224bd65047c54c075fa117585"
server_key_version: "encryptionkey@0"
I20260808 22:35:55.423794 382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786228291013016-32094-0/minicluster-data/master-0/wal/instance:
uuid: "9332021a27a947c2aae7db392e9b2910"
format_stamp: "Formatted at 2026-08-08 22:35:55 on dist-test-slave-266d"
server_key: "83287fb7f65c79fe301fc8dfaba267a1"
server_key_iv: "911cf64224bd65047c54c075fa117585"
server_key_version: "encryptionkey@0"
I20260808 22:35:55.427443 382 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 22:35:55.429705 410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:55.430783 382 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:35:55.430902 382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "9332021a27a947c2aae7db392e9b2910"
format_stamp: "Formatted at 2026-08-08 22:35:55 on dist-test-slave-266d"
server_key: "83287fb7f65c79fe301fc8dfaba267a1"
server_key_iv: "911cf64224bd65047c54c075fa117585"
server_key_version: "encryptionkey@0"
I20260808 22:35:55.430995 382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:35:55.451468 382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:35:55.454020 382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:35:55.454207 382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:35:55.461133 382 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38267
I20260808 22:35:55.461156 490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38267 every 8 connection(s)
I20260808 22:35:55.462054 382 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:35:55.464561 491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:35:55.465001 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 382
I20260808 22:35:55.465179 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786228291013016-32094-0/minicluster-data/master-0/wal/instance
I20260808 22:35:55.465451 32094 external_mini_cluster.cc:1428] Setting key a902559ddc7653d41a35e2f581884d8b
I20260808 22:35:55.470799 491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9332021a27a947c2aae7db392e9b2910: Bootstrap starting.
I20260808 22:35:55.473173 491 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9332021a27a947c2aae7db392e9b2910: Neither blocks nor log segments found. Creating new log.
I20260808 22:35:55.473848 491 log.cc:824] T 00000000000000000000000000000000 P 9332021a27a947c2aae7db392e9b2910: Log is configured to *not* fsync() on all Append() calls
Aug 08 22:35:55 dist-test-slave-266d krb5kdc[364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228555, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 22:35:55.475817 491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9332021a27a947c2aae7db392e9b2910: No bootstrap required, opened a new log
I20260808 22:35:55.479290 498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:55.466769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.165:60929 (local address 127.0.0.1:38267)
0808 22:35:55.467150 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:55.467168 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:55.467210 (+ 42us) server_negotiation.cc:407] Connection header received
0808 22:35:55.468007 (+ 797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:55.468030 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:55.468479 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:55.468831 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:55.469505 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:55.470935 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:55.471754 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:55.472430 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:55.472579 (+ 149us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:55.475064 (+ 2485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:55.475069 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:55.475081 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:35:55.475106 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:55.476647 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:35:55.477176 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:35:55.477180 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:35:55.477184 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:35:55.477286 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:35:55.477600 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:35:55.477620 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:35:55.477621 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:35:55.477955 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:55.478109 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:55.478395 (+ 286us) server_negotiation.cc:299] Negotiation successful
0808 22:35:55.478582 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":100,"threads_started":1}
I20260808 22:35:55.479441 491 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9332021a27a947c2aae7db392e9b2910 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9332021a27a947c2aae7db392e9b2910" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38267 } }
I20260808 22:35:55.479658 491 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9332021a27a947c2aae7db392e9b2910 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:35:55.479719 491 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9332021a27a947c2aae7db392e9b2910 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9332021a27a947c2aae7db392e9b2910, State: Initialized, Role: FOLLOWER
I20260808 22:35:55.480108 491 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9332021a27a947c2aae7db392e9b2910 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9332021a27a947c2aae7db392e9b2910" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38267 } }
I20260808 22:35:55.480228 491 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9332021a27a947c2aae7db392e9b2910 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:35:55.480309 491 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9332021a27a947c2aae7db392e9b2910 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:35:55.480409 491 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9332021a27a947c2aae7db392e9b2910 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:35:55.481415 491 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9332021a27a947c2aae7db392e9b2910 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9332021a27a947c2aae7db392e9b2910" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38267 } }
I20260808 22:35:55.481709 491 leader_election.cc:302] T 00000000000000000000000000000000 P 9332021a27a947c2aae7db392e9b2910 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9332021a27a947c2aae7db392e9b2910; no voters:
I20260808 22:35:55.481966 491 leader_election.cc:288] T 00000000000000000000000000000000 P 9332021a27a947c2aae7db392e9b2910 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:35:55.482043 502 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9332021a27a947c2aae7db392e9b2910 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:35:55.482139 32094 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:35:55.482236 502 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9332021a27a947c2aae7db392e9b2910 [term 1 LEADER]: Becoming Leader. State: Replica: 9332021a27a947c2aae7db392e9b2910, State: Running, Role: LEADER
I20260808 22:35:55.482579 502 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9332021a27a947c2aae7db392e9b2910 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9332021a27a947c2aae7db392e9b2910" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38267 } }
I20260808 22:35:55.483160 491 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9332021a27a947c2aae7db392e9b2910 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:35:55.484272 505 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9332021a27a947c2aae7db392e9b2910 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9332021a27a947c2aae7db392e9b2910" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9332021a27a947c2aae7db392e9b2910" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38267 } } }
I20260808 22:35:55.484306 504 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9332021a27a947c2aae7db392e9b2910 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9332021a27a947c2aae7db392e9b2910. Latest consensus state: current_term: 1 leader_uuid: "9332021a27a947c2aae7db392e9b2910" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9332021a27a947c2aae7db392e9b2910" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38267 } } }
I20260808 22:35:55.484411 504 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9332021a27a947c2aae7db392e9b2910 [sys.catalog]: This master's current role is: LEADER
I20260808 22:35:55.484395 505 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9332021a27a947c2aae7db392e9b2910 [sys.catalog]: This master's current role is: LEADER
I20260808 22:35:55.484867 525 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:35:55.487874 525 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:35:55.493072 525 catalog_manager.cc:1347] Generated new cluster ID: d83b77e65ddb4efcb942f888012f62e5
I20260808 22:35:55.493152 525 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:35:55.497717 498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:55.484360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.165:47601 (local address 127.0.0.1:38267)
0808 22:35:55.484515 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:55.484519 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:55.484628 (+ 109us) server_negotiation.cc:407] Connection header received
0808 22:35:55.484802 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:55.484805 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:55.484847 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:55.484894 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:55.485809 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:55.486515 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:55.487597 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:55.487912 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:55.488016 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:55.489548 (+ 1532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:55.489552 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:55.489555 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:35:55.489580 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:55.491811 (+ 2231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:35:55.492300 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:35:55.492303 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:35:55.492305 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:35:55.492387 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:35:55.492722 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:35:55.492725 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:35:55.492727 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:35:55.492922 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:55.493060 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:55.497485 (+ 4425us) server_negotiation.cc:299] Negotiation successful
0808 22:35:55.497577 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260808 22:35:55.532281 525 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:35:55.533499 525 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:35:55.540822 525 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9332021a27a947c2aae7db392e9b2910: Generated new TSK 0
I20260808 22:35:55.541574 525 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:35:55.550628 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 382
W20260808 22:35:55.567250 361 connection.cc:535] client connection to 127.0.0.1:38267 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38267): Connection reset by peer (error 104)
2026-08-08T22:35:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1533 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786228291013016-32094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:35:55 dist-test-slave-266d krb5kdc[541](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:35:55 dist-test-slave-266d krb5kdc[541](info): set up 2 sockets
Aug 08 22:35:55 dist-test-slave-266d krb5kdc[541](info): commencing operation
krb5kdc: starting...
W20260808 22:35:56.717077 32094 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Aug 08 22:35:56 dist-test-slave-266d krb5kdc[541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228556, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:35:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:35:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786228291013016-32094-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786228291013016-32094-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 22:35:56.833245 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46611
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786228291013016-32094-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46611
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:35:56.941751 566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:35:56.941977 566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:35:56.942022 566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:35:56.945894 566 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:35:56.945945 566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:35:56.945976 566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:35:56.946005 566 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:35:56.946030 566 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:35:56.951041 566 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786228291013016-32094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46611
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786228291013016-32094-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.566
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:35:56.951948 566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:35:56.952883 566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:35:56.960748 574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:56.960764 573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:56.960728 576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:56.960906 566 server_base.cc:1034] running on GCE node
I20260808 22:35:56.961540 566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:35:56.962530 566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:35:56.963697 566 hybrid_clock.cc:648] HybridClock initialized: now 1786228556963686 us; error 25 us; skew 500 ppm
Aug 08 22:35:56 dist-test-slave-266d krb5kdc[541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228556, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:35:56.966253 566 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 22:35:56.967031 566 webserver.cc:460] Webserver started at http://127.0.0.1:39373/ using document root <none> and password file <none>
I20260808 22:35:56.967540 566 fs_manager.cc:359] Metadata directory not provided
I20260808 22:35:56.967602 566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:35:56.967765 566 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:35:56.969660 566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786228291013016-32094-0/minicluster-data/master-0/data/instance:
uuid: "84aa37543cb546869ced2c1535418f10"
format_stamp: "Formatted at 2026-08-08 22:35:56 on dist-test-slave-266d"
server_key: "671141fb9054c1b4952ea2f1fb3108bb"
server_key_iv: "0889382b71c7f32a97d6676c641c3df3"
server_key_version: "encryptionkey@0"
I20260808 22:35:56.970173 566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786228291013016-32094-0/minicluster-data/master-0/wal/instance:
uuid: "84aa37543cb546869ced2c1535418f10"
format_stamp: "Formatted at 2026-08-08 22:35:56 on dist-test-slave-266d"
server_key: "671141fb9054c1b4952ea2f1fb3108bb"
server_key_iv: "0889382b71c7f32a97d6676c641c3df3"
server_key_version: "encryptionkey@0"
I20260808 22:35:56.973582 566 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:35:56.975734 588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:56.976826 566 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:35:56.976924 566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "84aa37543cb546869ced2c1535418f10"
format_stamp: "Formatted at 2026-08-08 22:35:56 on dist-test-slave-266d"
server_key: "671141fb9054c1b4952ea2f1fb3108bb"
server_key_iv: "0889382b71c7f32a97d6676c641c3df3"
server_key_version: "encryptionkey@0"
I20260808 22:35:56.977012 566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:35:56.991402 566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:35:56.993994 566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:35:56.994150 566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:35:57.000969 566 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46611
I20260808 22:35:57.001034 666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46611 every 8 connection(s)
I20260808 22:35:57.001886 566 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:35:57.004498 667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:35:57.008134 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 566
I20260808 22:35:57.008273 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786228291013016-32094-0/minicluster-data/master-0/wal/instance
I20260808 22:35:57.008529 32094 external_mini_cluster.cc:1428] Setting key 4d3b6bd1ba7eeb9ebf0488dbd11b2291
I20260808 22:35:57.009928 667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84aa37543cb546869ced2c1535418f10: Bootstrap starting.
I20260808 22:35:57.012578 667 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84aa37543cb546869ced2c1535418f10: Neither blocks nor log segments found. Creating new log.
I20260808 22:35:57.013453 667 log.cc:824] T 00000000000000000000000000000000 P 84aa37543cb546869ced2c1535418f10: Log is configured to *not* fsync() on all Append() calls
I20260808 22:35:57.015359 667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84aa37543cb546869ced2c1535418f10: No bootstrap required, opened a new log
I20260808 22:35:57.018302 672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:57.009890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.165:57211 (local address 127.0.0.1:46611)
0808 22:35:57.010296 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:57.010315 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:57.010356 (+ 41us) server_negotiation.cc:407] Connection header received
0808 22:35:57.011180 (+ 824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:57.011205 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:57.011630 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:57.011972 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:57.012378 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:57.013860 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:57.014795 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:57.015462 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:57.015627 (+ 165us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:57.016376 (+ 749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:57.016382 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:57.016397 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:35:57.016857 (+ 460us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:57.016922 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:35:57.016952 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:57.017112 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:57.017251 (+ 139us) server_negotiation.cc:299] Negotiation successful
0808 22:35:57.017381 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":113,"threads_started":1}
I20260808 22:35:57.019461 667 raft_consensus.cc:348] T 00000000000000000000000000000000 P 84aa37543cb546869ced2c1535418f10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84aa37543cb546869ced2c1535418f10" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46611 } }
I20260808 22:35:57.019726 667 raft_consensus.cc:374] T 00000000000000000000000000000000 P 84aa37543cb546869ced2c1535418f10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:35:57.019802 667 raft_consensus.cc:729] T 00000000000000000000000000000000 P 84aa37543cb546869ced2c1535418f10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84aa37543cb546869ced2c1535418f10, State: Initialized, Role: FOLLOWER
I20260808 22:35:57.020210 667 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84aa37543cb546869ced2c1535418f10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84aa37543cb546869ced2c1535418f10" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46611 } }
I20260808 22:35:57.020364 667 raft_consensus.cc:388] T 00000000000000000000000000000000 P 84aa37543cb546869ced2c1535418f10 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:35:57.020435 667 raft_consensus.cc:482] T 00000000000000000000000000000000 P 84aa37543cb546869ced2c1535418f10 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:35:57.020533 667 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84aa37543cb546869ced2c1535418f10 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:35:57.021149 32094 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:35:57.021550 667 raft_consensus.cc:504] T 00000000000000000000000000000000 P 84aa37543cb546869ced2c1535418f10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84aa37543cb546869ced2c1535418f10" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46611 } }
I20260808 22:35:57.021915 667 leader_election.cc:302] T 00000000000000000000000000000000 P 84aa37543cb546869ced2c1535418f10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 84aa37543cb546869ced2c1535418f10; no voters:
I20260808 22:35:57.022238 667 leader_election.cc:288] T 00000000000000000000000000000000 P 84aa37543cb546869ced2c1535418f10 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:35:57.022321 679 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 84aa37543cb546869ced2c1535418f10 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:35:57.022568 679 raft_consensus.cc:686] T 00000000000000000000000000000000 P 84aa37543cb546869ced2c1535418f10 [term 1 LEADER]: Becoming Leader. State: Replica: 84aa37543cb546869ced2c1535418f10, State: Running, Role: LEADER
I20260808 22:35:57.022914 679 consensus_queue.cc:237] T 00000000000000000000000000000000 P 84aa37543cb546869ced2c1535418f10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84aa37543cb546869ced2c1535418f10" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46611 } }
I20260808 22:35:57.023463 667 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84aa37543cb546869ced2c1535418f10 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:35:57.024650 687 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84aa37543cb546869ced2c1535418f10 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 84aa37543cb546869ced2c1535418f10. Latest consensus state: current_term: 1 leader_uuid: "84aa37543cb546869ced2c1535418f10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84aa37543cb546869ced2c1535418f10" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46611 } } }
I20260808 22:35:57.024780 687 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84aa37543cb546869ced2c1535418f10 [sys.catalog]: This master's current role is: LEADER
I20260808 22:35:57.025076 684 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84aa37543cb546869ced2c1535418f10 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "84aa37543cb546869ced2c1535418f10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84aa37543cb546869ced2c1535418f10" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46611 } } }
I20260808 22:35:57.025184 684 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84aa37543cb546869ced2c1535418f10 [sys.catalog]: This master's current role is: LEADER
I20260808 22:35:57.027840 672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:57.023311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.165:39897 (local address 127.0.0.1:46611)
0808 22:35:57.023405 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:57.023409 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:57.023731 (+ 322us) server_negotiation.cc:407] Connection header received
0808 22:35:57.023848 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:57.023851 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:57.023896 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:57.023970 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:57.024352 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:57.025051 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:57.026368 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:57.026593 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:57.026679 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:57.027431 (+ 752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:57.027435 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:57.027438 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:35:57.027450 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:57.027474 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:35:57.027484 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:57.027554 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:57.027671 (+ 117us) server_negotiation.cc:299] Negotiation successful
0808 22:35:57.027704 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
W20260808 22:35:57.028348 712 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 84aa37543cb546869ced2c1535418f10: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:35:57.028429 712 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:35:57.028765 705 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:35:57.029682 705 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:35:57.035097 705 catalog_manager.cc:1347] Generated new cluster ID: 7b3563972d194dabb6a0dc8644e1d04c
I20260808 22:35:57.035148 705 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:35:57.047232 705 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:35:57.048045 705 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:35:57.058276 705 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 84aa37543cb546869ced2c1535418f10: Generated new TSK 0
I20260808 22:35:57.058835 705 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:35:57.065410 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 566
W20260808 22:35:57.077116 538 connection.cc:535] client connection to 127.0.0.1:46611 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46611): Connection reset by peer (error 104)
2026-08-08T22:35:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1509 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786228291013016-32094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:35:57 dist-test-slave-266d krb5kdc[723](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:35:57 dist-test-slave-266d krb5kdc[723](info): set up 2 sockets
Aug 08 22:35:57 dist-test-slave-266d krb5kdc[723](info): commencing operation
krb5kdc: starting...
W20260808 22:35:58.206816 32094 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Aug 08 22:35:58 dist-test-slave-266d krb5kdc[723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228558, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:35:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:35:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786228291013016-32094-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786228291013016-32094-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 22:35:58.323123 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33121
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786228291013016-32094-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33121
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:35:58.428937 738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:35:58.429221 738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:35:58.429294 738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:35:58.433497 738 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:35:58.433584 738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:35:58.433620 738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:35:58.433648 738 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:35:58.433683 738 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:35:58.438814 738 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786228291013016-32094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33121
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786228291013016-32094-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.738
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:35:58.439739 738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:35:58.440665 738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:35:58.448638 751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:58.448654 748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:58.448760 738 server_base.cc:1034] running on GCE node
W20260808 22:35:58.448652 747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:58.449491 738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:35:58.450522 738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:35:58.451704 738 hybrid_clock.cc:648] HybridClock initialized: now 1786228558451674 us; error 41 us; skew 500 ppm
Aug 08 22:35:58 dist-test-slave-266d krb5kdc[723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228558, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:35:58.456979 738 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 22:35:58.457931 738 webserver.cc:460] Webserver started at http://127.0.0.1:37869/ using document root <none> and password file <none>
I20260808 22:35:58.458482 738 fs_manager.cc:359] Metadata directory not provided
I20260808 22:35:58.458549 738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:35:58.458781 738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:35:58.460826 738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786228291013016-32094-0/minicluster-data/master-0/data/instance:
uuid: "32027cd3ca0b4c2e85513b98a2d2b995"
format_stamp: "Formatted at 2026-08-08 22:35:58 on dist-test-slave-266d"
server_key: "e2dd6b8523dd9c28df326d430b804648"
server_key_iv: "32943a118918eed3b096feae47ddb5dc"
server_key_version: "encryptionkey@0"
I20260808 22:35:58.461374 738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786228291013016-32094-0/minicluster-data/master-0/wal/instance:
uuid: "32027cd3ca0b4c2e85513b98a2d2b995"
format_stamp: "Formatted at 2026-08-08 22:35:58 on dist-test-slave-266d"
server_key: "e2dd6b8523dd9c28df326d430b804648"
server_key_iv: "32943a118918eed3b096feae47ddb5dc"
server_key_version: "encryptionkey@0"
I20260808 22:35:58.464995 738 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 22:35:58.467342 760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:58.468544 738 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:35:58.468672 738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "32027cd3ca0b4c2e85513b98a2d2b995"
format_stamp: "Formatted at 2026-08-08 22:35:58 on dist-test-slave-266d"
server_key: "e2dd6b8523dd9c28df326d430b804648"
server_key_iv: "32943a118918eed3b096feae47ddb5dc"
server_key_version: "encryptionkey@0"
I20260808 22:35:58.468760 738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:35:58.473028 738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:35:58.475651 738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:35:58.475786 738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:35:58.482347 738 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33121
I20260808 22:35:58.482357 838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33121 every 8 connection(s)
I20260808 22:35:58.483198 738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:35:58.485761 840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:35:58.487977 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 738
I20260808 22:35:58.488098 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786228291013016-32094-0/minicluster-data/master-0/wal/instance
I20260808 22:35:58.488381 32094 external_mini_cluster.cc:1428] Setting key c8f741af09f7b602f518476921aa6c62
I20260808 22:35:58.491617 840 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32027cd3ca0b4c2e85513b98a2d2b995: Bootstrap starting.
I20260808 22:35:58.493634 845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:58.489700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.165:38319 (local address 127.0.0.1:33121)
0808 22:35:58.490106 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:58.490128 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:58.490169 (+ 41us) server_negotiation.cc:407] Connection header received
0808 22:35:58.490973 (+ 804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:58.490996 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:58.491329 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:58.491673 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:58.492288 (+ 615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:58.492294 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:58.492307 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:35:58.492319 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:58.492374 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:35:58.492398 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:58.492526 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:58.492592 (+ 66us) server_negotiation.cc:299] Negotiation successful
0808 22:35:58.492746 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":122,"threads_started":1}
I20260808 22:35:58.494192 840 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 32027cd3ca0b4c2e85513b98a2d2b995: Neither blocks nor log segments found. Creating new log.
I20260808 22:35:58.494994 840 log.cc:824] T 00000000000000000000000000000000 P 32027cd3ca0b4c2e85513b98a2d2b995: Log is configured to *not* fsync() on all Append() calls
I20260808 22:35:58.496296 32094 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:35:58.497068 840 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32027cd3ca0b4c2e85513b98a2d2b995: No bootstrap required, opened a new log
I20260808 22:35:58.499730 845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:58.498257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.165:47155 (local address 127.0.0.1:33121)
0808 22:35:58.498352 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:58.498356 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:58.498539 (+ 183us) server_negotiation.cc:407] Connection header received
0808 22:35:58.498666 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:58.498669 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:58.498701 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:58.498761 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:58.499399 (+ 638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:58.499402 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:58.499405 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:35:58.499416 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:58.499441 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:35:58.499451 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:58.499519 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:58.499590 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 22:35:58.499628 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 22:35:58.500949 840 raft_consensus.cc:348] T 00000000000000000000000000000000 P 32027cd3ca0b4c2e85513b98a2d2b995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32027cd3ca0b4c2e85513b98a2d2b995" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33121 } }
I20260808 22:35:58.501179 840 raft_consensus.cc:374] T 00000000000000000000000000000000 P 32027cd3ca0b4c2e85513b98a2d2b995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:35:58.501237 840 raft_consensus.cc:729] T 00000000000000000000000000000000 P 32027cd3ca0b4c2e85513b98a2d2b995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32027cd3ca0b4c2e85513b98a2d2b995, State: Initialized, Role: FOLLOWER
I20260808 22:35:58.501614 840 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32027cd3ca0b4c2e85513b98a2d2b995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32027cd3ca0b4c2e85513b98a2d2b995" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33121 } }
I20260808 22:35:58.501751 840 raft_consensus.cc:388] T 00000000000000000000000000000000 P 32027cd3ca0b4c2e85513b98a2d2b995 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:35:58.501813 840 raft_consensus.cc:482] T 00000000000000000000000000000000 P 32027cd3ca0b4c2e85513b98a2d2b995 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:35:58.501910 840 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 32027cd3ca0b4c2e85513b98a2d2b995 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:35:58.502710 840 raft_consensus.cc:504] T 00000000000000000000000000000000 P 32027cd3ca0b4c2e85513b98a2d2b995 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32027cd3ca0b4c2e85513b98a2d2b995" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33121 } }
I20260808 22:35:58.503014 840 leader_election.cc:302] T 00000000000000000000000000000000 P 32027cd3ca0b4c2e85513b98a2d2b995 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 32027cd3ca0b4c2e85513b98a2d2b995; no voters:
I20260808 22:35:58.503268 840 leader_election.cc:288] T 00000000000000000000000000000000 P 32027cd3ca0b4c2e85513b98a2d2b995 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:35:58.503355 856 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 32027cd3ca0b4c2e85513b98a2d2b995 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:35:58.503530 856 raft_consensus.cc:686] T 00000000000000000000000000000000 P 32027cd3ca0b4c2e85513b98a2d2b995 [term 1 LEADER]: Becoming Leader. State: Replica: 32027cd3ca0b4c2e85513b98a2d2b995, State: Running, Role: LEADER
I20260808 22:35:58.503850 856 consensus_queue.cc:237] T 00000000000000000000000000000000 P 32027cd3ca0b4c2e85513b98a2d2b995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32027cd3ca0b4c2e85513b98a2d2b995" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33121 } }
I20260808 22:35:58.504352 840 sys_catalog.cc:564] T 00000000000000000000000000000000 P 32027cd3ca0b4c2e85513b98a2d2b995 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:35:58.505349 858 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32027cd3ca0b4c2e85513b98a2d2b995 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 32027cd3ca0b4c2e85513b98a2d2b995. Latest consensus state: current_term: 1 leader_uuid: "32027cd3ca0b4c2e85513b98a2d2b995" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32027cd3ca0b4c2e85513b98a2d2b995" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33121 } } }
I20260808 22:35:58.505465 858 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32027cd3ca0b4c2e85513b98a2d2b995 [sys.catalog]: This master's current role is: LEADER
I20260808 22:35:58.505842 870 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:35:58.505867 857 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32027cd3ca0b4c2e85513b98a2d2b995 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "32027cd3ca0b4c2e85513b98a2d2b995" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32027cd3ca0b4c2e85513b98a2d2b995" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33121 } } }
I20260808 22:35:58.506701 857 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32027cd3ca0b4c2e85513b98a2d2b995 [sys.catalog]: This master's current role is: LEADER
I20260808 22:35:58.508623 870 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:35:58.513314 870 catalog_manager.cc:1347] Generated new cluster ID: 6009919b91484a3c8b2d560f7bc1ddf9
I20260808 22:35:58.513376 870 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:35:58.533269 870 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:35:58.533582 870 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:35:58.538865 870 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 32027cd3ca0b4c2e85513b98a2d2b995: Generated new TSK 0
I20260808 22:35:58.539397 870 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:35:58.547640 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 738
2026-08-08T22:35:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1481 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786228291013016-32094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:35:58 dist-test-slave-266d krb5kdc[890](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:35:58 dist-test-slave-266d krb5kdc[890](info): set up 2 sockets
Aug 08 22:35:58 dist-test-slave-266d krb5kdc[890](info): commencing operation
krb5kdc: starting...
W20260808 22:35:59.690474 32094 mini_kdc.cc:121] Time spent starting KDC: real 1.118s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Aug 08 22:35:59 dist-test-slave-266d krb5kdc[890](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228559, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:35:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:35:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.87.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
I20260808 22:35:59.810078 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:35559
--webserver_interface=127.31.87.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:42935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:35559
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:35:59.919116 916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:35:59.919364 916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:35:59.919418 916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:35:59.923273 916 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:35:59.923324 916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:35:59.923349 916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:35:59.923370 916 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:35:59.923398 916 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:35:59.928481 916 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:42935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786228291013016-32094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:35559
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:35559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.87.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.916
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:35:59.929436 916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:35:59.930382 916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:35:59.938346 928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:59.938393 916 server_base.cc:1034] running on GCE node
W20260808 22:35:59.938347 922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:35:59.938354 923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:35:59.939175 916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:35:59.940179 916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:35:59.941335 916 hybrid_clock.cc:648] HybridClock initialized: now 1786228559941323 us; error 23 us; skew 500 ppm
Aug 08 22:35:59 dist-test-slave-266d krb5kdc[890](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228559, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:35:59.943917 916 init.cc:377] Logged in from keytab as kudu/127.31.87.190@KRBTEST.COM (short username kudu)
I20260808 22:35:59.944716 916 webserver.cc:460] Webserver started at http://127.31.87.190:37985/ using document root <none> and password file <none>
I20260808 22:35:59.945281 916 fs_manager.cc:359] Metadata directory not provided
I20260808 22:35:59.945346 916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:35:59.945544 916 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:35:59.947448 916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786228291013016-32094-0/minicluster-data/master-0/data/instance:
uuid: "1c708f37f5814398863d54ff174f8d30"
format_stamp: "Formatted at 2026-08-08 22:35:59 on dist-test-slave-266d"
server_key: "895b461fc497b0c770ac6602876ae9a4"
server_key_iv: "b9714a0d46be9082537f866cf98b7b1a"
server_key_version: "encryptionkey@0"
I20260808 22:35:59.947963 916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786228291013016-32094-0/minicluster-data/master-0/wal/instance:
uuid: "1c708f37f5814398863d54ff174f8d30"
format_stamp: "Formatted at 2026-08-08 22:35:59 on dist-test-slave-266d"
server_key: "895b461fc497b0c770ac6602876ae9a4"
server_key_iv: "b9714a0d46be9082537f866cf98b7b1a"
server_key_version: "encryptionkey@0"
I20260808 22:35:59.951406 916 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260808 22:35:59.953562 941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:35:59.954663 916 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:35:59.954770 916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "1c708f37f5814398863d54ff174f8d30"
format_stamp: "Formatted at 2026-08-08 22:35:59 on dist-test-slave-266d"
server_key: "895b461fc497b0c770ac6602876ae9a4"
server_key_iv: "b9714a0d46be9082537f866cf98b7b1a"
server_key_version: "encryptionkey@0"
I20260808 22:35:59.954857 916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:35:59.965807 916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:35:59.968223 916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:35:59.968386 916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:35:59.974917 916 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.190:35559
I20260808 22:35:59.974927 1035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.190:35559 every 8 connection(s)
I20260808 22:35:59.975770 916 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:35:59.978377 1037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:35:59.983376 1037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c708f37f5814398863d54ff174f8d30: Bootstrap starting.
I20260808 22:35:59.985641 1037 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c708f37f5814398863d54ff174f8d30: Neither blocks nor log segments found. Creating new log.
I20260808 22:35:59.985852 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 916
I20260808 22:35:59.985985 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786228291013016-32094-0/minicluster-data/master-0/wal/instance
I20260808 22:35:59.986228 32094 external_mini_cluster.cc:1428] Setting key a3716c35eebd9aed5a864c28ad40c38e
I20260808 22:35:59.986351 1037 log.cc:824] T 00000000000000000000000000000000 P 1c708f37f5814398863d54ff174f8d30: Log is configured to *not* fsync() on all Append() calls
I20260808 22:35:59.988435 1037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c708f37f5814398863d54ff174f8d30: No bootstrap required, opened a new log
I20260808 22:35:59.991629 1037 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c708f37f5814398863d54ff174f8d30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c708f37f5814398863d54ff174f8d30" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 35559 } }
I20260808 22:35:59.991829 1037 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c708f37f5814398863d54ff174f8d30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:35:59.991922 1037 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c708f37f5814398863d54ff174f8d30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c708f37f5814398863d54ff174f8d30, State: Initialized, Role: FOLLOWER
I20260808 22:35:59.992376 1037 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c708f37f5814398863d54ff174f8d30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c708f37f5814398863d54ff174f8d30" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 35559 } }
I20260808 22:35:59.992533 1037 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c708f37f5814398863d54ff174f8d30 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:35:59.992604 1037 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c708f37f5814398863d54ff174f8d30 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:35:59.992707 1037 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c708f37f5814398863d54ff174f8d30 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:35:59.993729 1037 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c708f37f5814398863d54ff174f8d30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c708f37f5814398863d54ff174f8d30" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 35559 } }
Aug 08 22:35:59 dist-test-slave-266d krb5kdc[890](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228559, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:35:59.994022 1037 leader_election.cc:302] T 00000000000000000000000000000000 P 1c708f37f5814398863d54ff174f8d30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c708f37f5814398863d54ff174f8d30; no voters:
I20260808 22:35:59.994277 1037 leader_election.cc:288] T 00000000000000000000000000000000 P 1c708f37f5814398863d54ff174f8d30 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:35:59.994372 1049 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c708f37f5814398863d54ff174f8d30 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:35:59.994602 1049 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c708f37f5814398863d54ff174f8d30 [term 1 LEADER]: Becoming Leader. State: Replica: 1c708f37f5814398863d54ff174f8d30, State: Running, Role: LEADER
I20260808 22:35:59.994946 1049 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c708f37f5814398863d54ff174f8d30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c708f37f5814398863d54ff174f8d30" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 35559 } }
I20260808 22:35:59.995421 1037 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c708f37f5814398863d54ff174f8d30 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:35:59.996547 1052 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c708f37f5814398863d54ff174f8d30 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c708f37f5814398863d54ff174f8d30. Latest consensus state: current_term: 1 leader_uuid: "1c708f37f5814398863d54ff174f8d30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c708f37f5814398863d54ff174f8d30" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 35559 } } }
I20260808 22:35:59.996670 1052 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c708f37f5814398863d54ff174f8d30 [sys.catalog]: This master's current role is: LEADER
I20260808 22:35:59.996990 1050 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c708f37f5814398863d54ff174f8d30 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1c708f37f5814398863d54ff174f8d30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c708f37f5814398863d54ff174f8d30" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 35559 } } }
I20260808 22:35:59.997114 1050 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c708f37f5814398863d54ff174f8d30 [sys.catalog]: This master's current role is: LEADER
I20260808 22:35:59.997448 1060 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:36:00.000059 1048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:35:59.987225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46193 (local address 127.31.87.190:35559)
0808 22:35:59.987560 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0808 22:35:59.987574 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 22:35:59.987607 (+ 33us) server_negotiation.cc:407] Connection header received
0808 22:35:59.988191 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:35:59.988219 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:35:59.988519 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:35:59.988824 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:35:59.989625 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:59.990542 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:59.991368 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:35:59.991804 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:35:59.991946 (+ 142us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:35:59.994748 (+ 2802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:35:59.994754 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:35:59.994769 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:35:59.994799 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:35:59.996780 (+ 1981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:35:59.997430 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:35:59.997436 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:35:59.997442 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:35:59.997600 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:35:59.997998 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:35:59.998002 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:35:59.998004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:35:59.998485 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:35:59.998721 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0808 22:35:59.998967 (+ 246us) server_negotiation.cc:299] Negotiation successful
0808 22:35:59.999181 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":79,"threads_started":1}
I20260808 22:36:00.001053 1060 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:36:00.003111 32094 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:36:00.007097 1060 catalog_manager.cc:1347] Generated new cluster ID: d7b9633c616b41dea3b09dcaba908d08
I20260808 22:36:00.007153 1060 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:36:00.013715 1048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:00.004905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59071 (local address 127.31.87.190:35559)
0808 22:36:00.005009 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:00.005014 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:00.005298 (+ 284us) server_negotiation.cc:407] Connection header received
0808 22:36:00.005404 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:00.005407 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:00.005452 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:00.005534 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:00.006318 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:00.007092 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:00.008274 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:00.008622 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:00.008715 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:00.010306 (+ 1591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:36:00.010309 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:36:00.010312 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:36:00.010335 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:36:00.011884 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:00.012420 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:00.012424 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:00.012425 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:00.012504 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:00.012812 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:00.012814 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:00.012816 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:00.013028 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:36:00.013207 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0808 22:36:00.013468 (+ 261us) server_negotiation.cc:299] Negotiation successful
0808 22:36:00.013552 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":37}
I20260808 22:36:00.014874 1060 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:36:00.015944 1060 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:36:00.035151 1060 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1c708f37f5814398863d54ff174f8d30: Generated new TSK 0
I20260808 22:36:00.035794 1060 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:36:00.041072 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 916
W20260808 22:36:00.061980 887 connection.cc:535] client connection to 127.31.87.190:35559 recv error: Network error: failed to read from TLS socket (remote: 127.31.87.190:35559): Connection reset by peer (error 104)
2026-08-08T22:36:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1503 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786228291013016-32094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:36:00 dist-test-slave-266d krb5kdc[1089](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:36:00 dist-test-slave-266d krb5kdc[1089](info): set up 2 sockets
Aug 08 22:36:00 dist-test-slave-266d krb5kdc[1089](info): commencing operation
krb5kdc: starting...
W20260808 22:36:01.212798 32094 mini_kdc.cc:121] Time spent starting KDC: real 1.137s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Aug 08 22:36:01 dist-test-slave-266d krb5kdc[1089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228561, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:36:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:36:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.87.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
I20260808 22:36:01.328402 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:39159
--webserver_interface=127.31.87.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:43585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:39159
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:36:01.434350 1117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:36:01.434567 1117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:36:01.434605 1117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:36:01.438452 1117 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:36:01.438498 1117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:36:01.438513 1117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:36:01.438534 1117 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:36:01.438550 1117 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:36:01.443424 1117 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:43585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786228291013016-32094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:39159
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:39159
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.87.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1117
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:36:01.444257 1117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:36:01.445139 1117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:36:01.452847 1127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:36:01.452972 1117 server_base.cc:1034] running on GCE node
W20260808 22:36:01.452876 1125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:36:01.452857 1129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:36:01.453619 1117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:36:01.454638 1117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:36:01.455791 1117 hybrid_clock.cc:648] HybridClock initialized: now 1786228561455766 us; error 37 us; skew 500 ppm
Aug 08 22:36:01 dist-test-slave-266d krb5kdc[1089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228561, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:36:01.458359 1117 init.cc:377] Logged in from keytab as kudu/127.31.87.190@KRBTEST.COM (short username kudu)
I20260808 22:36:01.459174 1117 webserver.cc:460] Webserver started at http://127.31.87.190:33907/ using document root <none> and password file <none>
I20260808 22:36:01.459672 1117 fs_manager.cc:359] Metadata directory not provided
I20260808 22:36:01.459736 1117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:36:01.459931 1117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:36:01.461870 1117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786228291013016-32094-0/minicluster-data/master-0/data/instance:
uuid: "61fa700dcbea4a08870cc028564fb0df"
format_stamp: "Formatted at 2026-08-08 22:36:01 on dist-test-slave-266d"
server_key: "c2929175346e5a49b7da869dfc628354"
server_key_iv: "14df0c3b8ca10fa2fc2dbdc13c76ec37"
server_key_version: "encryptionkey@0"
I20260808 22:36:01.462385 1117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786228291013016-32094-0/minicluster-data/master-0/wal/instance:
uuid: "61fa700dcbea4a08870cc028564fb0df"
format_stamp: "Formatted at 2026-08-08 22:36:01 on dist-test-slave-266d"
server_key: "c2929175346e5a49b7da869dfc628354"
server_key_iv: "14df0c3b8ca10fa2fc2dbdc13c76ec37"
server_key_version: "encryptionkey@0"
I20260808 22:36:01.465819 1117 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:36:01.467957 1137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:36:01.469030 1117 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:36:01.469169 1117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "61fa700dcbea4a08870cc028564fb0df"
format_stamp: "Formatted at 2026-08-08 22:36:01 on dist-test-slave-266d"
server_key: "c2929175346e5a49b7da869dfc628354"
server_key_iv: "14df0c3b8ca10fa2fc2dbdc13c76ec37"
server_key_version: "encryptionkey@0"
I20260808 22:36:01.469257 1117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:36:01.488291 1117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:36:01.490666 1117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:36:01.490823 1117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:36:01.497077 1117 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.190:39159
I20260808 22:36:01.497113 1210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.190:39159 every 8 connection(s)
I20260808 22:36:01.497956 1117 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:36:01.500415 1212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:36:01.503551 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 1117
I20260808 22:36:01.503710 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786228291013016-32094-0/minicluster-data/master-0/wal/instance
I20260808 22:36:01.503952 32094 external_mini_cluster.cc:1428] Setting key e8b8bb5f1e4470639df0acb7d648a97e
I20260808 22:36:01.506212 1212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61fa700dcbea4a08870cc028564fb0df: Bootstrap starting.
I20260808 22:36:01.508797 1212 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 61fa700dcbea4a08870cc028564fb0df: Neither blocks nor log segments found. Creating new log.
I20260808 22:36:01.509573 1212 log.cc:824] T 00000000000000000000000000000000 P 61fa700dcbea4a08870cc028564fb0df: Log is configured to *not* fsync() on all Append() calls
I20260808 22:36:01.511583 1212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61fa700dcbea4a08870cc028564fb0df: No bootstrap required, opened a new log
I20260808 22:36:01.513324 1218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:01.505235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56221 (local address 127.31.87.190:39159)
0808 22:36:01.505633 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:01.505651 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:01.505692 (+ 41us) server_negotiation.cc:407] Connection header received
0808 22:36:01.506471 (+ 779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:01.506496 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:01.506890 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:01.507229 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:01.507648 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:01.509053 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:01.509980 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:01.510547 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:01.510697 (+ 150us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:01.511457 (+ 760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:36:01.511464 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:36:01.511479 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:36:01.511940 (+ 461us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:36:01.512004 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:36:01.512035 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:36:01.512168 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 22:36:01.512318 (+ 150us) server_negotiation.cc:299] Negotiation successful
0808 22:36:01.512441 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":110,"threads_started":1}
I20260808 22:36:01.515615 1212 raft_consensus.cc:348] T 00000000000000000000000000000000 P 61fa700dcbea4a08870cc028564fb0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61fa700dcbea4a08870cc028564fb0df" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 39159 } }
I20260808 22:36:01.515874 1212 raft_consensus.cc:374] T 00000000000000000000000000000000 P 61fa700dcbea4a08870cc028564fb0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:36:01.515939 1212 raft_consensus.cc:729] T 00000000000000000000000000000000 P 61fa700dcbea4a08870cc028564fb0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61fa700dcbea4a08870cc028564fb0df, State: Initialized, Role: FOLLOWER
I20260808 22:36:01.516173 32094 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:36:01.516372 1212 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61fa700dcbea4a08870cc028564fb0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61fa700dcbea4a08870cc028564fb0df" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 39159 } }
I20260808 22:36:01.516503 1212 raft_consensus.cc:388] T 00000000000000000000000000000000 P 61fa700dcbea4a08870cc028564fb0df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:36:01.516566 1212 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61fa700dcbea4a08870cc028564fb0df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:36:01.516659 1212 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61fa700dcbea4a08870cc028564fb0df [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:36:01.517535 1212 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61fa700dcbea4a08870cc028564fb0df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61fa700dcbea4a08870cc028564fb0df" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 39159 } }
I20260808 22:36:01.517848 1212 leader_election.cc:302] T 00000000000000000000000000000000 P 61fa700dcbea4a08870cc028564fb0df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 61fa700dcbea4a08870cc028564fb0df; no voters:
I20260808 22:36:01.518136 1212 leader_election.cc:288] T 00000000000000000000000000000000 P 61fa700dcbea4a08870cc028564fb0df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:36:01.518229 1231 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 61fa700dcbea4a08870cc028564fb0df [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:36:01.518448 1231 raft_consensus.cc:686] T 00000000000000000000000000000000 P 61fa700dcbea4a08870cc028564fb0df [term 1 LEADER]: Becoming Leader. State: Replica: 61fa700dcbea4a08870cc028564fb0df, State: Running, Role: LEADER
I20260808 22:36:01.518746 1231 consensus_queue.cc:237] T 00000000000000000000000000000000 P 61fa700dcbea4a08870cc028564fb0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61fa700dcbea4a08870cc028564fb0df" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 39159 } }
I20260808 22:36:01.519289 1212 sys_catalog.cc:564] T 00000000000000000000000000000000 P 61fa700dcbea4a08870cc028564fb0df [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:36:01.520272 1235 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61fa700dcbea4a08870cc028564fb0df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 61fa700dcbea4a08870cc028564fb0df. Latest consensus state: current_term: 1 leader_uuid: "61fa700dcbea4a08870cc028564fb0df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61fa700dcbea4a08870cc028564fb0df" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 39159 } } }
I20260808 22:36:01.520393 1235 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61fa700dcbea4a08870cc028564fb0df [sys.catalog]: This master's current role is: LEADER
I20260808 22:36:01.521713 1234 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61fa700dcbea4a08870cc028564fb0df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "61fa700dcbea4a08870cc028564fb0df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61fa700dcbea4a08870cc028564fb0df" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 39159 } } }
I20260808 22:36:01.521855 1234 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61fa700dcbea4a08870cc028564fb0df [sys.catalog]: This master's current role is: LEADER
I20260808 22:36:01.524202 1218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:01.518087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37339 (local address 127.31.87.190:39159)
0808 22:36:01.518190 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:01.518194 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:01.520146 (+ 1952us) server_negotiation.cc:407] Connection header received
0808 22:36:01.520324 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:01.520327 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:01.520369 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:01.520418 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:01.520774 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:01.521377 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:01.522436 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:01.522763 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:01.522835 (+ 72us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:01.523813 (+ 978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:36:01.523816 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:36:01.523819 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:36:01.523831 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:36:01.523856 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:36:01.523868 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:36:01.523964 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 22:36:01.524058 (+ 94us) server_negotiation.cc:299] Negotiation successful
0808 22:36:01.524089 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260808 22:36:01.526019 1256 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 61fa700dcbea4a08870cc028564fb0df: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:36:01.526086 1256 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:36:01.526170 1240 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:36:01.527068 1240 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:36:01.532269 1240 catalog_manager.cc:1347] Generated new cluster ID: 6e8f40ed4a4a430bac33fc5703fd644e
I20260808 22:36:01.532318 1240 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:36:01.551604 1240 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:36:01.552433 1240 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:36:01.560559 1240 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 61fa700dcbea4a08870cc028564fb0df: Generated new TSK 0
I20260808 22:36:01.561169 1240 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:36:01.570641 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 1117
W20260808 22:36:01.594548 1087 connection.cc:535] client connection to 127.31.87.190:39159 recv error: Network error: failed to read from TLS socket (remote: 127.31.87.190:39159): Connection reset by peer (error 104)
2026-08-08T22:36:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1532 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786228291013016-32094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:36:01 dist-test-slave-266d krb5kdc[1264](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:36:01 dist-test-slave-266d krb5kdc[1264](info): set up 2 sockets
Aug 08 22:36:01 dist-test-slave-266d krb5kdc[1264](info): commencing operation
krb5kdc: starting...
W20260808 22:36:02.721055 32094 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Aug 08 22:36:02 dist-test-slave-266d krb5kdc[1264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228562, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:36:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:36:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.87.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
I20260808 22:36:02.835793 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:40249
--webserver_interface=127.31.87.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:34451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:40249
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:36:02.944473 1287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:36:02.944729 1287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:36:02.944777 1287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:36:02.948654 1287 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:36:02.948709 1287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:36:02.948731 1287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:36:02.948752 1287 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:36:02.948786 1287 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:36:02.953850 1287 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:34451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786228291013016-32094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:40249
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:40249
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.87.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:36:02.954810 1287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:36:02.955741 1287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:36:02.963891 1298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:36:02.963958 1295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:36:02.964074 1287 server_base.cc:1034] running on GCE node
W20260808 22:36:02.963901 1296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:36:02.964725 1287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:36:02.965766 1287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:36:02.966925 1287 hybrid_clock.cc:648] HybridClock initialized: now 1786228562966912 us; error 26 us; skew 500 ppm
Aug 08 22:36:02 dist-test-slave-266d krb5kdc[1264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228562, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:36:02.969440 1287 init.cc:377] Logged in from keytab as kudu/127.31.87.190@KRBTEST.COM (short username kudu)
I20260808 22:36:02.970254 1287 webserver.cc:460] Webserver started at http://127.31.87.190:34157/ using document root <none> and password file <none>
I20260808 22:36:02.970770 1287 fs_manager.cc:359] Metadata directory not provided
I20260808 22:36:02.970834 1287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:36:02.971014 1287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:36:02.972908 1287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786228291013016-32094-0/minicluster-data/master-0/data/instance:
uuid: "2f48cc484c2a4e1798a5c60cd103f7c6"
format_stamp: "Formatted at 2026-08-08 22:36:02 on dist-test-slave-266d"
server_key: "38dce30f3d5a20f8a5f8fbedb9c6b190"
server_key_iv: "113fbd0c8ffe97f031bfa1e1573b274f"
server_key_version: "encryptionkey@0"
I20260808 22:36:02.973467 1287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786228291013016-32094-0/minicluster-data/master-0/wal/instance:
uuid: "2f48cc484c2a4e1798a5c60cd103f7c6"
format_stamp: "Formatted at 2026-08-08 22:36:02 on dist-test-slave-266d"
server_key: "38dce30f3d5a20f8a5f8fbedb9c6b190"
server_key_iv: "113fbd0c8ffe97f031bfa1e1573b274f"
server_key_version: "encryptionkey@0"
I20260808 22:36:02.976891 1287 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 22:36:02.979064 1309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:36:02.980149 1287 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:36:02.980264 1287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "2f48cc484c2a4e1798a5c60cd103f7c6"
format_stamp: "Formatted at 2026-08-08 22:36:02 on dist-test-slave-266d"
server_key: "38dce30f3d5a20f8a5f8fbedb9c6b190"
server_key_iv: "113fbd0c8ffe97f031bfa1e1573b274f"
server_key_version: "encryptionkey@0"
I20260808 22:36:02.980355 1287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:36:02.984560 1287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:36:02.987082 1287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:36:02.987239 1287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:36:02.993412 1287 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.190:40249
I20260808 22:36:02.993455 1384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.190:40249 every 8 connection(s)
I20260808 22:36:02.994218 1287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:36:02.996695 1386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:36:03.000941 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 1287
I20260808 22:36:03.001113 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786228291013016-32094-0/minicluster-data/master-0/wal/instance
I20260808 22:36:03.001437 32094 external_mini_cluster.cc:1428] Setting key 12f6c92517700ad28fd2d1c793ec9bba
I20260808 22:36:03.002024 1386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f48cc484c2a4e1798a5c60cd103f7c6: Bootstrap starting.
I20260808 22:36:03.004376 1386 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f48cc484c2a4e1798a5c60cd103f7c6: Neither blocks nor log segments found. Creating new log.
I20260808 22:36:03.005345 1386 log.cc:824] T 00000000000000000000000000000000 P 2f48cc484c2a4e1798a5c60cd103f7c6: Log is configured to *not* fsync() on all Append() calls
I20260808 22:36:03.006565 1392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:03.002735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46421 (local address 127.31.87.190:40249)
0808 22:36:03.003163 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:03.003178 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:03.003211 (+ 33us) server_negotiation.cc:407] Connection header received
0808 22:36:03.003800 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:03.003816 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:03.004084 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:03.004432 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:03.005231 (+ 799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:36:03.005236 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:36:03.005252 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:36:03.005264 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:36:03.005329 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:36:03.005355 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:36:03.005455 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 22:36:03.005625 (+ 170us) server_negotiation.cc:299] Negotiation successful
0808 22:36:03.005761 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":126,"threads_started":1}
I20260808 22:36:03.007318 1386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f48cc484c2a4e1798a5c60cd103f7c6: No bootstrap required, opened a new log
I20260808 22:36:03.009271 32094 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:36:03.011592 1386 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f48cc484c2a4e1798a5c60cd103f7c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f48cc484c2a4e1798a5c60cd103f7c6" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 40249 } }
I20260808 22:36:03.011823 1386 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2f48cc484c2a4e1798a5c60cd103f7c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:36:03.011930 1386 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f48cc484c2a4e1798a5c60cd103f7c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f48cc484c2a4e1798a5c60cd103f7c6, State: Initialized, Role: FOLLOWER
I20260808 22:36:03.012385 1386 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f48cc484c2a4e1798a5c60cd103f7c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f48cc484c2a4e1798a5c60cd103f7c6" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 40249 } }
I20260808 22:36:03.012545 1386 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f48cc484c2a4e1798a5c60cd103f7c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:36:03.012660 1386 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f48cc484c2a4e1798a5c60cd103f7c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:36:03.012758 1386 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f48cc484c2a4e1798a5c60cd103f7c6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:36:03.013032 1392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:03.011437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48525 (local address 127.31.87.190:40249)
0808 22:36:03.011549 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:03.011553 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:03.011773 (+ 220us) server_negotiation.cc:407] Connection header received
0808 22:36:03.011876 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:03.011879 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:03.011914 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:03.011997 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:03.012636 (+ 639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:36:03.012640 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:36:03.012642 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:36:03.012653 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:36:03.012680 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:36:03.012691 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:36:03.012779 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 22:36:03.012893 (+ 114us) server_negotiation.cc:299] Negotiation successful
0808 22:36:03.012925 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 22:36:03.013885 1386 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f48cc484c2a4e1798a5c60cd103f7c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f48cc484c2a4e1798a5c60cd103f7c6" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 40249 } }
I20260808 22:36:03.014194 1386 leader_election.cc:302] T 00000000000000000000000000000000 P 2f48cc484c2a4e1798a5c60cd103f7c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f48cc484c2a4e1798a5c60cd103f7c6; no voters:
I20260808 22:36:03.014467 1386 leader_election.cc:288] T 00000000000000000000000000000000 P 2f48cc484c2a4e1798a5c60cd103f7c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:36:03.014536 1405 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f48cc484c2a4e1798a5c60cd103f7c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:36:03.014707 1405 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f48cc484c2a4e1798a5c60cd103f7c6 [term 1 LEADER]: Becoming Leader. State: Replica: 2f48cc484c2a4e1798a5c60cd103f7c6, State: Running, Role: LEADER
I20260808 22:36:03.015031 1405 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f48cc484c2a4e1798a5c60cd103f7c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f48cc484c2a4e1798a5c60cd103f7c6" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 40249 } }
I20260808 22:36:03.015560 1386 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f48cc484c2a4e1798a5c60cd103f7c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:36:03.016533 1411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f48cc484c2a4e1798a5c60cd103f7c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2f48cc484c2a4e1798a5c60cd103f7c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f48cc484c2a4e1798a5c60cd103f7c6" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 40249 } } }
I20260808 22:36:03.016660 1411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f48cc484c2a4e1798a5c60cd103f7c6 [sys.catalog]: This master's current role is: LEADER
I20260808 22:36:03.016927 1412 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f48cc484c2a4e1798a5c60cd103f7c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f48cc484c2a4e1798a5c60cd103f7c6. Latest consensus state: current_term: 1 leader_uuid: "2f48cc484c2a4e1798a5c60cd103f7c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f48cc484c2a4e1798a5c60cd103f7c6" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 40249 } } }
I20260808 22:36:03.017009 1412 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f48cc484c2a4e1798a5c60cd103f7c6 [sys.catalog]: This master's current role is: LEADER
I20260808 22:36:03.017021 1423 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:36:03.019913 1423 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:36:03.024946 1423 catalog_manager.cc:1347] Generated new cluster ID: 903ff234c857434c8ea21f80ab2b95a7
I20260808 22:36:03.025007 1423 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:36:03.045980 1423 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:36:03.046329 1423 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:36:03.054359 1423 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2f48cc484c2a4e1798a5c60cd103f7c6: Generated new TSK 0
I20260808 22:36:03.054975 1423 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:36:03.066046 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 1287
2026-08-08T22:36:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1486 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786228291013016-32094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:36:03 dist-test-slave-266d krb5kdc[1447](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:36:03 dist-test-slave-266d krb5kdc[1447](info): set up 2 sockets
Aug 08 22:36:03 dist-test-slave-266d krb5kdc[1447](info): commencing operation
krb5kdc: starting...
W20260808 22:36:04.207669 32094 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Aug 08 22:36:04 dist-test-slave-266d krb5kdc[1447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228564, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:36:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:36:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.87.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
I20260808 22:36:04.320900 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:35791
--webserver_interface=127.31.87.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:40641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:35791
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:36:04.426718 1469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:36:04.426944 1469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:36:04.426999 1469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:36:04.430868 1469 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:36:04.430922 1469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:36:04.430948 1469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:36:04.430969 1469 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:36:04.430998 1469 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:36:04.435897 1469 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:40641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786228291013016-32094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:35791
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:35791
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.87.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:36:04.436803 1469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:36:04.437741 1469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:36:04.445479 1475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:36:04.445541 1479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:36:04.445508 1484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:36:04.445677 1469 server_base.cc:1034] running on GCE node
I20260808 22:36:04.446183 1469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:36:04.447154 1469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:36:04.448310 1469 hybrid_clock.cc:648] HybridClock initialized: now 1786228564448296 us; error 30 us; skew 500 ppm
Aug 08 22:36:04 dist-test-slave-266d krb5kdc[1447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228564, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:36:04.455461 1469 init.cc:377] Logged in from keytab as kudu/127.31.87.190@KRBTEST.COM (short username kudu)
I20260808 22:36:04.456240 1469 webserver.cc:460] Webserver started at http://127.31.87.190:36427/ using document root <none> and password file <none>
I20260808 22:36:04.456753 1469 fs_manager.cc:359] Metadata directory not provided
I20260808 22:36:04.456820 1469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:36:04.457005 1469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:36:04.458889 1469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786228291013016-32094-0/minicluster-data/master-0/data/instance:
uuid: "be3d525fa2d946e6b903d0ab655dbd34"
format_stamp: "Formatted at 2026-08-08 22:36:04 on dist-test-slave-266d"
server_key: "395c4ff9d56abe9167391caac62452de"
server_key_iv: "17f128accbd611a2737c396f61d3c768"
server_key_version: "encryptionkey@0"
I20260808 22:36:04.459404 1469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786228291013016-32094-0/minicluster-data/master-0/wal/instance:
uuid: "be3d525fa2d946e6b903d0ab655dbd34"
format_stamp: "Formatted at 2026-08-08 22:36:04 on dist-test-slave-266d"
server_key: "395c4ff9d56abe9167391caac62452de"
server_key_iv: "17f128accbd611a2737c396f61d3c768"
server_key_version: "encryptionkey@0"
I20260808 22:36:04.462800 1469 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 22:36:04.464896 1504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:36:04.466039 1469 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:36:04.466145 1469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "be3d525fa2d946e6b903d0ab655dbd34"
format_stamp: "Formatted at 2026-08-08 22:36:04 on dist-test-slave-266d"
server_key: "395c4ff9d56abe9167391caac62452de"
server_key_iv: "17f128accbd611a2737c396f61d3c768"
server_key_version: "encryptionkey@0"
I20260808 22:36:04.466234 1469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:36:04.477757 1469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:36:04.480165 1469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:36:04.480327 1469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:36:04.486778 1469 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.190:35791
I20260808 22:36:04.486788 1587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.190:35791 every 8 connection(s)
I20260808 22:36:04.487608 1469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:36:04.490168 1589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:36:04.495340 1589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be3d525fa2d946e6b903d0ab655dbd34: Bootstrap starting.
I20260808 22:36:04.495688 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 1469
I20260808 22:36:04.495826 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786228291013016-32094-0/minicluster-data/master-0/wal/instance
I20260808 22:36:04.496060 32094 external_mini_cluster.cc:1428] Setting key 137665d3ff4094bb4d133680ec0e78f4
I20260808 22:36:04.497812 1589 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P be3d525fa2d946e6b903d0ab655dbd34: Neither blocks nor log segments found. Creating new log.
I20260808 22:36:04.498672 1589 log.cc:824] T 00000000000000000000000000000000 P be3d525fa2d946e6b903d0ab655dbd34: Log is configured to *not* fsync() on all Append() calls
I20260808 22:36:04.500656 1589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be3d525fa2d946e6b903d0ab655dbd34: No bootstrap required, opened a new log
I20260808 22:36:04.504410 1589 raft_consensus.cc:348] T 00000000000000000000000000000000 P be3d525fa2d946e6b903d0ab655dbd34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be3d525fa2d946e6b903d0ab655dbd34" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 35791 } }
I20260808 22:36:04.504673 1589 raft_consensus.cc:374] T 00000000000000000000000000000000 P be3d525fa2d946e6b903d0ab655dbd34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:36:04.504732 1589 raft_consensus.cc:729] T 00000000000000000000000000000000 P be3d525fa2d946e6b903d0ab655dbd34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be3d525fa2d946e6b903d0ab655dbd34, State: Initialized, Role: FOLLOWER
Aug 08 22:36:04 dist-test-slave-266d krb5kdc[1447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228564, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:36:04.505214 1589 consensus_queue.cc:260] T 00000000000000000000000000000000 P be3d525fa2d946e6b903d0ab655dbd34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be3d525fa2d946e6b903d0ab655dbd34" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 35791 } }
I20260808 22:36:04.505381 1589 raft_consensus.cc:388] T 00000000000000000000000000000000 P be3d525fa2d946e6b903d0ab655dbd34 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:36:04.505450 1589 raft_consensus.cc:482] T 00000000000000000000000000000000 P be3d525fa2d946e6b903d0ab655dbd34 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:36:04.505556 1589 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be3d525fa2d946e6b903d0ab655dbd34 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:36:04.506456 1589 raft_consensus.cc:504] T 00000000000000000000000000000000 P be3d525fa2d946e6b903d0ab655dbd34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be3d525fa2d946e6b903d0ab655dbd34" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 35791 } }
I20260808 22:36:04.506780 1589 leader_election.cc:302] T 00000000000000000000000000000000 P be3d525fa2d946e6b903d0ab655dbd34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: be3d525fa2d946e6b903d0ab655dbd34; no voters:
I20260808 22:36:04.507045 1589 leader_election.cc:288] T 00000000000000000000000000000000 P be3d525fa2d946e6b903d0ab655dbd34 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:36:04.507136 1596 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be3d525fa2d946e6b903d0ab655dbd34 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:36:04.507296 1596 raft_consensus.cc:686] T 00000000000000000000000000000000 P be3d525fa2d946e6b903d0ab655dbd34 [term 1 LEADER]: Becoming Leader. State: Replica: be3d525fa2d946e6b903d0ab655dbd34, State: Running, Role: LEADER
I20260808 22:36:04.507632 1596 consensus_queue.cc:237] T 00000000000000000000000000000000 P be3d525fa2d946e6b903d0ab655dbd34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be3d525fa2d946e6b903d0ab655dbd34" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 35791 } }
I20260808 22:36:04.508257 1589 sys_catalog.cc:564] T 00000000000000000000000000000000 P be3d525fa2d946e6b903d0ab655dbd34 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:36:04.509268 1600 sys_catalog.cc:455] T 00000000000000000000000000000000 P be3d525fa2d946e6b903d0ab655dbd34 [sys.catalog]: SysCatalogTable state changed. Reason: New leader be3d525fa2d946e6b903d0ab655dbd34. Latest consensus state: current_term: 1 leader_uuid: "be3d525fa2d946e6b903d0ab655dbd34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be3d525fa2d946e6b903d0ab655dbd34" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 35791 } } }
I20260808 22:36:04.509389 1600 sys_catalog.cc:458] T 00000000000000000000000000000000 P be3d525fa2d946e6b903d0ab655dbd34 [sys.catalog]: This master's current role is: LEADER
I20260808 22:36:04.509789 1602 sys_catalog.cc:455] T 00000000000000000000000000000000 P be3d525fa2d946e6b903d0ab655dbd34 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "be3d525fa2d946e6b903d0ab655dbd34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be3d525fa2d946e6b903d0ab655dbd34" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 35791 } } }
I20260808 22:36:04.509909 1602 sys_catalog.cc:458] T 00000000000000000000000000000000 P be3d525fa2d946e6b903d0ab655dbd34 [sys.catalog]: This master's current role is: LEADER
I20260808 22:36:04.510924 1593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:04.497284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38311 (local address 127.31.87.190:35791)
0808 22:36:04.497682 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:04.497701 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:04.497741 (+ 40us) server_negotiation.cc:407] Connection header received
0808 22:36:04.498531 (+ 790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:04.498554 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:04.498957 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:04.499259 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:04.499978 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:04.501404 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:04.502222 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:04.502900 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:04.503065 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:04.505604 (+ 2539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:36:04.505611 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:36:04.505628 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:36:04.505659 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:36:04.507470 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:04.507939 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:04.507943 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:04.507947 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:04.508046 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:04.508298 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:04.508301 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:04.508303 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:04.508745 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:36:04.509507 (+ 762us) server_negotiation.cc:1035] Waiting for connection context
0808 22:36:04.509662 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 22:36:04.510017 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":92,"threads_started":1}
W20260808 22:36:04.513449 1624 catalog_manager.cc:1558] T 00000000000000000000000000000000 P be3d525fa2d946e6b903d0ab655dbd34: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:36:04.513514 1624 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:36:04.513859 1611 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:36:04.514777 1611 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:36:04.516661 32094 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:36:04.521018 1611 catalog_manager.cc:1347] Generated new cluster ID: f31052142d0646da9862814335444a3c
I20260808 22:36:04.521157 1611 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:36:04.527541 1593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:04.518767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50425 (local address 127.31.87.190:35791)
0808 22:36:04.518889 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:04.518893 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:04.519542 (+ 649us) server_negotiation.cc:407] Connection header received
0808 22:36:04.519712 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:04.519715 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:04.519756 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:04.519822 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:04.520510 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:04.521230 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:04.522024 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:04.522385 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:04.522484 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:04.523830 (+ 1346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:36:04.523834 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:36:04.523837 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:36:04.523864 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:36:04.525284 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:04.525698 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:04.525702 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:04.525704 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:04.525787 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:04.526149 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:04.526153 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:04.526154 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:04.526508 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:36:04.526768 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0808 22:36:04.527264 (+ 496us) server_negotiation.cc:299] Negotiation successful
0808 22:36:04.527373 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 22:36:04.532083 1611 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:36:04.532976 1611 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:36:04.544467 1611 catalog_manager.cc:5868] T 00000000000000000000000000000000 P be3d525fa2d946e6b903d0ab655dbd34: Generated new TSK 0
I20260808 22:36:04.545147 1611 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:36:04.552554 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 1469
2026-08-08T22:36:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1486 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786228291013016-32094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:36:04 dist-test-slave-266d krb5kdc[1642](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:36:04 dist-test-slave-266d krb5kdc[1642](info): set up 2 sockets
Aug 08 22:36:04 dist-test-slave-266d krb5kdc[1642](info): commencing operation
krb5kdc: starting...
W20260808 22:36:05.689997 32094 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Aug 08 22:36:05 dist-test-slave-266d krb5kdc[1642](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:36:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:36:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.87.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
I20260808 22:36:05.804916 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:43045
--webserver_interface=127.31.87.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:36627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:43045
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:36:05.915813 1662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:36:05.916081 1662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:36:05.916146 1662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:36:05.920102 1662 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:36:05.920171 1662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:36:05.920203 1662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:36:05.920230 1662 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:36:05.920262 1662 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:36:05.925513 1662 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:36627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786228291013016-32094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:43045
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:43045
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.87.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1662
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:36:05.926494 1662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:36:05.927450 1662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:36:05.935714 1671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:36:05.935770 1670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:36:05.935828 1662 server_base.cc:1034] running on GCE node
W20260808 22:36:05.935714 1674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:36:05.936518 1662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:36:05.937587 1662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:36:05.938750 1662 hybrid_clock.cc:648] HybridClock initialized: now 1786228565938734 us; error 25 us; skew 500 ppm
Aug 08 22:36:05 dist-test-slave-266d krb5kdc[1642](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228565, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:36:05.941397 1662 init.cc:377] Logged in from keytab as kudu/127.31.87.190@KRBTEST.COM (short username kudu)
I20260808 22:36:05.942279 1662 webserver.cc:460] Webserver started at http://127.31.87.190:40963/ using document root <none> and password file <none>
I20260808 22:36:05.942813 1662 fs_manager.cc:359] Metadata directory not provided
I20260808 22:36:05.942880 1662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:36:05.943096 1662 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:36:05.945145 1662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786228291013016-32094-0/minicluster-data/master-0/data/instance:
uuid: "1b4e74000e6240ffb17483ce9771c864"
format_stamp: "Formatted at 2026-08-08 22:36:05 on dist-test-slave-266d"
server_key: "6f4430c0a9402842f7f8c7a727826ea2"
server_key_iv: "6ed2a88b19071375ea8457fe501e4667"
server_key_version: "encryptionkey@0"
I20260808 22:36:05.945745 1662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786228291013016-32094-0/minicluster-data/master-0/wal/instance:
uuid: "1b4e74000e6240ffb17483ce9771c864"
format_stamp: "Formatted at 2026-08-08 22:36:05 on dist-test-slave-266d"
server_key: "6f4430c0a9402842f7f8c7a727826ea2"
server_key_iv: "6ed2a88b19071375ea8457fe501e4667"
server_key_version: "encryptionkey@0"
I20260808 22:36:05.949371 1662 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 22:36:05.951705 1688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:36:05.952845 1662 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:36:05.952956 1662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "1b4e74000e6240ffb17483ce9771c864"
format_stamp: "Formatted at 2026-08-08 22:36:05 on dist-test-slave-266d"
server_key: "6f4430c0a9402842f7f8c7a727826ea2"
server_key_iv: "6ed2a88b19071375ea8457fe501e4667"
server_key_version: "encryptionkey@0"
I20260808 22:36:05.953051 1662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:36:05.968855 1662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:36:05.971951 1662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:36:05.972129 1662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:36:05.979162 1662 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.190:43045
I20260808 22:36:05.979168 1772 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.190:43045 every 8 connection(s)
I20260808 22:36:05.980109 1662 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:36:05.982748 1773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:36:05.987955 1773 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b4e74000e6240ffb17483ce9771c864: Bootstrap starting.
I20260808 22:36:05.990227 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 1662
I20260808 22:36:05.990345 1773 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b4e74000e6240ffb17483ce9771c864: Neither blocks nor log segments found. Creating new log.
I20260808 22:36:05.990374 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786228291013016-32094-0/minicluster-data/master-0/wal/instance
I20260808 22:36:05.990679 32094 external_mini_cluster.cc:1428] Setting key 456e1aea836a0268ddd2ed8d0da84488
I20260808 22:36:05.991179 1773 log.cc:824] T 00000000000000000000000000000000 P 1b4e74000e6240ffb17483ce9771c864: Log is configured to *not* fsync() on all Append() calls
I20260808 22:36:05.993032 1773 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b4e74000e6240ffb17483ce9771c864: No bootstrap required, opened a new log
I20260808 22:36:05.996271 1773 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b4e74000e6240ffb17483ce9771c864 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b4e74000e6240ffb17483ce9771c864" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 43045 } }
I20260808 22:36:05.996505 1773 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b4e74000e6240ffb17483ce9771c864 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:36:05.996564 1773 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b4e74000e6240ffb17483ce9771c864 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b4e74000e6240ffb17483ce9771c864, State: Initialized, Role: FOLLOWER
I20260808 22:36:05.997008 1773 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b4e74000e6240ffb17483ce9771c864 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b4e74000e6240ffb17483ce9771c864" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 43045 } }
I20260808 22:36:05.997171 1773 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b4e74000e6240ffb17483ce9771c864 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:36:05.997236 1773 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b4e74000e6240ffb17483ce9771c864 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:36:05.997334 1773 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b4e74000e6240ffb17483ce9771c864 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:36:05.998199 1773 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b4e74000e6240ffb17483ce9771c864 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b4e74000e6240ffb17483ce9771c864" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 43045 } }
I20260808 22:36:05.998504 1773 leader_election.cc:302] T 00000000000000000000000000000000 P 1b4e74000e6240ffb17483ce9771c864 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b4e74000e6240ffb17483ce9771c864; no voters:
I20260808 22:36:05.998662 1780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:05.991738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51033 (local address 127.31.87.190:43045)
0808 22:36:05.992105 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:05.992125 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:05.992157 (+ 32us) server_negotiation.cc:407] Connection header received
0808 22:36:05.992737 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:05.992752 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:05.993052 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:05.993385 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:05.993755 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:05.994712 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:05.995576 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:05.996021 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:05.996162 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:05.996962 (+ 800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:36:05.996967 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:36:05.996979 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:36:05.997447 (+ 468us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:36:05.997509 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:36:05.997533 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:36:05.997617 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 22:36:05.997749 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 22:36:05.997944 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":117,"threads_started":1}
I20260808 22:36:05.998795 1773 leader_election.cc:288] T 00000000000000000000000000000000 P 1b4e74000e6240ffb17483ce9771c864 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:36:05.998960 1781 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b4e74000e6240ffb17483ce9771c864 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:36:05.999243 1781 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b4e74000e6240ffb17483ce9771c864 [term 1 LEADER]: Becoming Leader. State: Replica: 1b4e74000e6240ffb17483ce9771c864, State: Running, Role: LEADER
I20260808 22:36:05.999571 1781 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b4e74000e6240ffb17483ce9771c864 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b4e74000e6240ffb17483ce9771c864" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 43045 } }
I20260808 22:36:05.999933 1773 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b4e74000e6240ffb17483ce9771c864 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:36:06.001230 1782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b4e74000e6240ffb17483ce9771c864 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1b4e74000e6240ffb17483ce9771c864" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b4e74000e6240ffb17483ce9771c864" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 43045 } } }
I20260808 22:36:06.001354 1782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b4e74000e6240ffb17483ce9771c864 [sys.catalog]: This master's current role is: LEADER
I20260808 22:36:06.001573 1783 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b4e74000e6240ffb17483ce9771c864 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b4e74000e6240ffb17483ce9771c864. Latest consensus state: current_term: 1 leader_uuid: "1b4e74000e6240ffb17483ce9771c864" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b4e74000e6240ffb17483ce9771c864" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 43045 } } }
I20260808 22:36:06.001654 1783 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b4e74000e6240ffb17483ce9771c864 [sys.catalog]: This master's current role is: LEADER
I20260808 22:36:06.001799 32094 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260808 22:36:06.005653 1802 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1b4e74000e6240ffb17483ce9771c864: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:36:06.005734 1802 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:36:06.005795 1796 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:36:06.006587 1796 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:36:06.008214 1780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:06.003941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40725 (local address 127.31.87.190:43045)
0808 22:36:06.004033 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:06.004037 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:06.004198 (+ 161us) server_negotiation.cc:407] Connection header received
0808 22:36:06.004309 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:06.004311 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:06.004354 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:06.004432 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:06.004704 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:06.005479 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:06.006522 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:06.006865 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:06.006972 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:06.007799 (+ 827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:36:06.007804 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:36:06.007807 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:36:06.007820 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:36:06.007848 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:36:06.007862 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:36:06.007936 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 22:36:06.008035 (+ 99us) server_negotiation.cc:299] Negotiation successful
0808 22:36:06.008087 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 22:36:06.012104 1796 catalog_manager.cc:1347] Generated new cluster ID: 414541884d8f494f9a39264c034717f6
I20260808 22:36:06.012158 1796 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:36:06.035495 1796 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:36:06.036721 1796 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:36:06.046813 1796 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b4e74000e6240ffb17483ce9771c864: Generated new TSK 0
I20260808 22:36:06.047578 1796 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:36:06.052534 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 1662
2026-08-08T22:36:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1503 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786228291013016-32094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:36:06 dist-test-slave-266d krb5kdc[1827](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:36:06 dist-test-slave-266d krb5kdc[1827](info): set up 2 sockets
Aug 08 22:36:06 dist-test-slave-266d krb5kdc[1827](info): commencing operation
krb5kdc: starting...
W20260808 22:36:07.199890 32094 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Aug 08 22:36:07 dist-test-slave-266d krb5kdc[1827](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:36:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:36:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.87.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
I20260808 22:36:07.318248 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:44785
--webserver_interface=127.31.87.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:42679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:44785
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:36:07.424140 1846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:36:07.424372 1846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:36:07.424427 1846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:36:07.428268 1846 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:36:07.428321 1846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:36:07.428347 1846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:36:07.428368 1846 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:36:07.428390 1846 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:36:07.433328 1846 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:42679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786228291013016-32094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:44785
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:44785
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.87.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1846
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:36:07.434239 1846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:36:07.435153 1846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:36:07.442939 1865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:36:07.442981 1861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:36:07.443023 1863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:36:07.443295 1846 server_base.cc:1034] running on GCE node
I20260808 22:36:07.443789 1846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:36:07.444766 1846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:36:07.445922 1846 hybrid_clock.cc:648] HybridClock initialized: now 1786228567445908 us; error 30 us; skew 500 ppm
Aug 08 22:36:07 dist-test-slave-266d krb5kdc[1827](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228567, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:36:07.448375 1846 init.cc:377] Logged in from keytab as kudu/127.31.87.190@KRBTEST.COM (short username kudu)
I20260808 22:36:07.449226 1846 webserver.cc:460] Webserver started at http://127.31.87.190:45319/ using document root <none> and password file <none>
I20260808 22:36:07.449730 1846 fs_manager.cc:359] Metadata directory not provided
I20260808 22:36:07.449796 1846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:36:07.449997 1846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:36:07.451905 1846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786228291013016-32094-0/minicluster-data/master-0/data/instance:
uuid: "58d9bb43115449a5bf6651b4d386cca4"
format_stamp: "Formatted at 2026-08-08 22:36:07 on dist-test-slave-266d"
server_key: "e6a0e0cc0efedc9b20e149d38e3c10d9"
server_key_iv: "d1522ea260602127f59d098fb8820107"
server_key_version: "encryptionkey@0"
I20260808 22:36:07.452421 1846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786228291013016-32094-0/minicluster-data/master-0/wal/instance:
uuid: "58d9bb43115449a5bf6651b4d386cca4"
format_stamp: "Formatted at 2026-08-08 22:36:07 on dist-test-slave-266d"
server_key: "e6a0e0cc0efedc9b20e149d38e3c10d9"
server_key_iv: "d1522ea260602127f59d098fb8820107"
server_key_version: "encryptionkey@0"
I20260808 22:36:07.455822 1846 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260808 22:36:07.458103 1878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:36:07.459239 1846 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:36:07.459353 1846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "58d9bb43115449a5bf6651b4d386cca4"
format_stamp: "Formatted at 2026-08-08 22:36:07 on dist-test-slave-266d"
server_key: "e6a0e0cc0efedc9b20e149d38e3c10d9"
server_key_iv: "d1522ea260602127f59d098fb8820107"
server_key_version: "encryptionkey@0"
I20260808 22:36:07.459438 1846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:36:07.463696 1846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:36:07.466225 1846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:36:07.466379 1846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:36:07.472695 1846 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.190:44785
I20260808 22:36:07.472718 1974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.190:44785 every 8 connection(s)
I20260808 22:36:07.473543 1846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:36:07.476085 1975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:36:07.481563 1975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58d9bb43115449a5bf6651b4d386cca4: Bootstrap starting.
I20260808 22:36:07.483546 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 1846
I20260808 22:36:07.483686 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786228291013016-32094-0/minicluster-data/master-0/wal/instance
I20260808 22:36:07.483969 32094 external_mini_cluster.cc:1428] Setting key cc8acae624d4f6b10acb63f9a4163af3
I20260808 22:36:07.484071 1975 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 58d9bb43115449a5bf6651b4d386cca4: Neither blocks nor log segments found. Creating new log.
I20260808 22:36:07.484903 1975 log.cc:824] T 00000000000000000000000000000000 P 58d9bb43115449a5bf6651b4d386cca4: Log is configured to *not* fsync() on all Append() calls
I20260808 22:36:07.486791 1975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58d9bb43115449a5bf6651b4d386cca4: No bootstrap required, opened a new log
I20260808 22:36:07.488497 1982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:07.485026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34803 (local address 127.31.87.190:44785)
0808 22:36:07.485448 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:07.485466 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:07.485500 (+ 34us) server_negotiation.cc:407] Connection header received
0808 22:36:07.486082 (+ 582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:07.486098 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:07.486360 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:07.486654 (+ 294us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:07.487394 (+ 740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:36:07.487399 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:36:07.487410 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:36:07.487418 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:36:07.487462 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:36:07.487479 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:36:07.487557 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 22:36:07.487690 (+ 133us) server_negotiation.cc:299] Negotiation successful
0808 22:36:07.487820 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":102,"threads_started":1}
I20260808 22:36:07.490433 1975 raft_consensus.cc:348] T 00000000000000000000000000000000 P 58d9bb43115449a5bf6651b4d386cca4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58d9bb43115449a5bf6651b4d386cca4" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 44785 } }
I20260808 22:36:07.490638 1975 raft_consensus.cc:374] T 00000000000000000000000000000000 P 58d9bb43115449a5bf6651b4d386cca4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:36:07.490693 1975 raft_consensus.cc:729] T 00000000000000000000000000000000 P 58d9bb43115449a5bf6651b4d386cca4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58d9bb43115449a5bf6651b4d386cca4, State: Initialized, Role: FOLLOWER
I20260808 22:36:07.490981 32094 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:36:07.491083 1975 consensus_queue.cc:260] T 00000000000000000000000000000000 P 58d9bb43115449a5bf6651b4d386cca4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58d9bb43115449a5bf6651b4d386cca4" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 44785 } }
I20260808 22:36:07.491245 1975 raft_consensus.cc:388] T 00000000000000000000000000000000 P 58d9bb43115449a5bf6651b4d386cca4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:36:07.491303 1975 raft_consensus.cc:482] T 00000000000000000000000000000000 P 58d9bb43115449a5bf6651b4d386cca4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:36:07.491389 1975 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 58d9bb43115449a5bf6651b4d386cca4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:36:07.492409 1975 raft_consensus.cc:504] T 00000000000000000000000000000000 P 58d9bb43115449a5bf6651b4d386cca4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58d9bb43115449a5bf6651b4d386cca4" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 44785 } }
I20260808 22:36:07.492753 1975 leader_election.cc:302] T 00000000000000000000000000000000 P 58d9bb43115449a5bf6651b4d386cca4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 58d9bb43115449a5bf6651b4d386cca4; no voters:
I20260808 22:36:07.493058 1975 leader_election.cc:288] T 00000000000000000000000000000000 P 58d9bb43115449a5bf6651b4d386cca4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:36:07.493304 1989 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 58d9bb43115449a5bf6651b4d386cca4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:36:07.493528 1989 raft_consensus.cc:686] T 00000000000000000000000000000000 P 58d9bb43115449a5bf6651b4d386cca4 [term 1 LEADER]: Becoming Leader. State: Replica: 58d9bb43115449a5bf6651b4d386cca4, State: Running, Role: LEADER
I20260808 22:36:07.493871 1989 consensus_queue.cc:237] T 00000000000000000000000000000000 P 58d9bb43115449a5bf6651b4d386cca4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58d9bb43115449a5bf6651b4d386cca4" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 44785 } }
I20260808 22:36:07.494297 1975 sys_catalog.cc:564] T 00000000000000000000000000000000 P 58d9bb43115449a5bf6651b4d386cca4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:36:07.494823 1982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:07.493235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48285 (local address 127.31.87.190:44785)
0808 22:36:07.493340 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:07.493344 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:07.493543 (+ 199us) server_negotiation.cc:407] Connection header received
0808 22:36:07.493683 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:07.493686 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:07.493715 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:07.493760 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:07.494478 (+ 718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:36:07.494482 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:36:07.494485 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:36:07.494497 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:36:07.494522 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:36:07.494534 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:36:07.494602 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 22:36:07.494704 (+ 102us) server_negotiation.cc:299] Negotiation successful
0808 22:36:07.494739 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 22:36:07.495010 1989 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58d9bb43115449a5bf6651b4d386cca4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 58d9bb43115449a5bf6651b4d386cca4. Latest consensus state: current_term: 1 leader_uuid: "58d9bb43115449a5bf6651b4d386cca4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58d9bb43115449a5bf6651b4d386cca4" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 44785 } } }
I20260808 22:36:07.495751 1989 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58d9bb43115449a5bf6651b4d386cca4 [sys.catalog]: This master's current role is: LEADER
I20260808 22:36:07.495637 1994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58d9bb43115449a5bf6651b4d386cca4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "58d9bb43115449a5bf6651b4d386cca4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58d9bb43115449a5bf6651b4d386cca4" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 44785 } } }
I20260808 22:36:07.496058 1994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58d9bb43115449a5bf6651b4d386cca4 [sys.catalog]: This master's current role is: LEADER
I20260808 22:36:07.496162 2013 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:36:07.499178 2013 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:36:07.504272 2013 catalog_manager.cc:1347] Generated new cluster ID: 3008f79cd689420b90fee2ad6cb6a1c0
I20260808 22:36:07.504324 2013 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:36:07.514714 2013 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:36:07.514973 2013 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:36:07.527272 2013 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 58d9bb43115449a5bf6651b4d386cca4: Generated new TSK 0
I20260808 22:36:07.527845 2013 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:36:07.535060 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 1846
2026-08-08T22:36:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1479 ms)
[----------] 18 tests from AuthTokenIssuingTest (26993 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:36:07 dist-test-slave-266d krb5kdc[2027](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:36:07 dist-test-slave-266d krb5kdc[2027](info): set up 2 sockets
Aug 08 22:36:07 dist-test-slave-266d krb5kdc[2027](info): commencing operation
krb5kdc: starting...
W20260808 22:36:08.705786 32094 mini_kdc.cc:121] Time spent starting KDC: real 1.142s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Aug 08 22:36:08 dist-test-slave-266d krb5kdc[2027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:36:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:36:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.87.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
I20260808 22:36:08.821900 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:44685
--webserver_interface=127.31.87.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:43083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:44685,127.31.87.189:33235,127.31.87.188:37639
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:36:08.927716 2047 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 22:36:08.927958 2047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:36:08.927983 2047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:36:08.928041 2047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:36:08.931910 2047 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:36:08.931960 2047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:36:08.931978 2047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:36:08.931996 2047 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:36:08.932025 2047 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:36:08.937074 2047 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:43083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:44685,127.31.87.189:33235,127.31.87.188:37639
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:44685
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.87.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2047
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:36:08.938037 2047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:36:08.938992 2047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:36:08.947121 2057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:36:08.947098 2059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:36:08.947162 2064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:36:08.947427 2047 server_base.cc:1034] running on GCE node
I20260808 22:36:08.947971 2047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:36:08.948983 2047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:36:08.950139 2047 hybrid_clock.cc:648] HybridClock initialized: now 1786228568950125 us; error 26 us; skew 500 ppm
Aug 08 22:36:08 dist-test-slave-266d krb5kdc[2027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228568, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:36:08.952662 2047 init.cc:377] Logged in from keytab as kudu/127.31.87.190@KRBTEST.COM (short username kudu)
I20260808 22:36:08.953506 2047 webserver.cc:460] Webserver started at http://127.31.87.190:35609/ using document root <none> and password file <none>
I20260808 22:36:08.954017 2047 fs_manager.cc:359] Metadata directory not provided
I20260808 22:36:08.954082 2047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:36:08.954290 2047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:36:08.956467 2047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/minicluster-data/master-0/data/instance:
uuid: "23ca781363d048a5a98b2e5db8a9f370"
format_stamp: "Formatted at 2026-08-08 22:36:08 on dist-test-slave-266d"
server_key: "dc6685f7184663df784c2b692a4ebd26"
server_key_iv: "2b4268a1e2540edaa9cd81bf9ee5fc0b"
server_key_version: "encryptionkey@0"
I20260808 22:36:08.957000 2047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/minicluster-data/master-0/wal/instance:
uuid: "23ca781363d048a5a98b2e5db8a9f370"
format_stamp: "Formatted at 2026-08-08 22:36:08 on dist-test-slave-266d"
server_key: "dc6685f7184663df784c2b692a4ebd26"
server_key_iv: "2b4268a1e2540edaa9cd81bf9ee5fc0b"
server_key_version: "encryptionkey@0"
I20260808 22:36:08.960371 2047 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 22:36:08.962599 2078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:36:08.963758 2047 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:36:08.963868 2047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "23ca781363d048a5a98b2e5db8a9f370"
format_stamp: "Formatted at 2026-08-08 22:36:08 on dist-test-slave-266d"
server_key: "dc6685f7184663df784c2b692a4ebd26"
server_key_iv: "2b4268a1e2540edaa9cd81bf9ee5fc0b"
server_key_version: "encryptionkey@0"
I20260808 22:36:08.963955 2047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:36:08.978859 2047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:36:08.981292 2047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:36:08.981451 2047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:36:08.987794 2047 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.190:44685
I20260808 22:36:08.988078 2162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.190:44685 every 8 connection(s)
I20260808 22:36:08.988695 2047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:36:08.991257 2165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:36:08.994670 2165 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.87.190" port: 44685 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:36:08.997049 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 2047
I20260808 22:36:08.997198 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/minicluster-data/master-0/wal/instance
I20260808 22:36:08.997480 32094 external_mini_cluster.cc:1428] Setting key f64cafdd326c49f5526601430064970c
Aug 08 22:36:08 dist-test-slave-266d krb5kdc[2027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228568, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.87.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.189@KRBTEST.COM" created.
I20260808 22:36:09.008881 2173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:08.996712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:44685 (local address 127.0.0.1:43876)
0808 22:36:08.997306 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0808 22:36:08.997351 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 22:36:08.998145 (+ 794us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:36:08.998835 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:36:08.998840 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:36:08.999130 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:36:08.999467 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:36:08.999485 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:36:09.000586 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:36:09.000590 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:36:09.001647 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:36:09.001659 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:36:09.002254 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:36:09.002258 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:36:09.002384 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:09.002901 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:36:09.002921 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:36:09.004437 (+ 1516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:36:09.006165 (+ 1728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:36:09.006174 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:36:09.006183 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:36:09.006385 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:36:09.006660 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:36:09.006663 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:36:09.006665 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:36:09.006772 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:36:09.007311 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:36:09.007317 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:36:09.007446 (+ 129us) client_negotiation.cc:769] Sending connection context
0808 22:36:09.007587 (+ 141us) client_negotiation.cc:240] Negotiation successful
0808 22:36:09.007977 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":417,"mutex_wait_us":56,"spinlock_wait_cycles":27520,"thread_start_us":148,"threads_started":1}
I20260808 22:36:09.008881 2172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:08.996712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43876 (local address 127.31.87.190:44685)
0808 22:36:08.997147 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:08.997166 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:08.997464 (+ 298us) server_negotiation.cc:407] Connection header received
0808 22:36:08.998440 (+ 976us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:08.998445 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:08.998657 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:08.998852 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:08.999609 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:09.000465 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:09.001748 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:09.002097 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:09.002204 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:09.004565 (+ 2361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:36:09.004570 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:36:09.004579 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:36:09.004600 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:36:09.006080 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:09.006479 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:09.006485 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:09.006486 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:09.006579 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:09.006880 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:09.006883 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:09.006885 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:09.007210 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:36:09.007334 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 22:36:09.007663 (+ 329us) server_negotiation.cc:299] Negotiation successful
0808 22:36:09.007884 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":271,"thread_start_us":121,"threads_started":1}
I20260808 22:36:09.010998 2165 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.87.189" port: 33235 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:36:09.011665 2173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:09.011435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.189:33235 (local address 127.0.0.1:34680)
0808 22:36:09.011542 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0808 22:36:09.011607 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.189:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 22:36:09.011833 2083 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.87.189:33235: connect: Connection refused (error 111)
I20260808 22:36:09.012633 2173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:09.012442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.189:33235 (local address 127.0.0.1:34692)
0808 22:36:09.012539 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0808 22:36:09.012582 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.189:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 22:36:09.012758 2165 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.87.189:33235: Network error: Client connection negotiation failed: client connection to 127.31.87.189:33235: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.189.keytab.
Entry for principal kudu/127.31.87.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.189.keytab.
I20260808 22:36:09.023151 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.189:33235
--webserver_interface=127.31.87.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.189.keytab
--principal=kudu/127.31.87.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:43083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:44685,127.31.87.189:33235,127.31.87.188:37639
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 22:36:09.061980 2165 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.87.189" port: 33235 } attempt: 1
I20260808 22:36:09.062783 2173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:09.062504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.189:33235 (local address 127.0.0.1:34698)
0808 22:36:09.062645 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 22:36:09.062705 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.189:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 22:36:09.063584 2173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:09.063358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.189:33235 (local address 127.0.0.1:34704)
0808 22:36:09.063482 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 22:36:09.063526 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.189:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260808 22:36:09.063722 2165 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.87.189:33235: Network error: Client connection negotiation failed: client connection to 127.31.87.189:33235: connect: Connection refused (error 111)
W20260808 22:36:09.129977 2180 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 22:36:09.130245 2180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:36:09.130280 2180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:36:09.130340 2180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 22:36:09.131865 2165 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.87.189" port: 33235 } attempt: 2
I20260808 22:36:09.132664 2173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:09.132382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.189:33235 (local address 127.0.0.1:34718)
0808 22:36:09.132524 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 22:36:09.132586 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.189:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 22:36:09.133368 2173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:09.133157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.189:33235 (local address 127.0.0.1:34730)
0808 22:36:09.133252 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0808 22:36:09.133308 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.189:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 22:36:09.133474 2165 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.87.189:33235: Network error: Client connection negotiation failed: client connection to 127.31.87.189:33235: connect: Connection refused (error 111)
W20260808 22:36:09.134501 2180 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:36:09.134553 2180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:36:09.134580 2180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:36:09.134610 2180 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:36:09.134634 2180 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:36:09.139786 2180 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:43083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:44685,127.31.87.189:33235,127.31.87.188:37639
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.189.keytab
--principal=kudu/127.31.87.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.189:33235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.31.87.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2180
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:36:09.140784 2180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:36:09.141773 2180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:36:09.149603 2190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:36:09.149609 2188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:36:09.149628 2187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:36:09.149953 2180 server_base.cc:1034] running on GCE node
I20260808 22:36:09.150445 2180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:36:09.151412 2180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:36:09.152565 2180 hybrid_clock.cc:648] HybridClock initialized: now 1786228569152541 us; error 33 us; skew 500 ppm
Aug 08 22:36:09 dist-test-slave-266d krb5kdc[2027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228569, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:36:09.155139 2180 init.cc:377] Logged in from keytab as kudu/127.31.87.189@KRBTEST.COM (short username kudu)
I20260808 22:36:09.155925 2180 webserver.cc:460] Webserver started at http://127.31.87.189:41783/ using document root <none> and password file <none>
I20260808 22:36:09.156431 2180 fs_manager.cc:359] Metadata directory not provided
I20260808 22:36:09.156494 2180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:36:09.156680 2180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:36:09.158583 2180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/minicluster-data/master-1/data/instance:
uuid: "5595372de9c648ec9277eac84bc4fc28"
format_stamp: "Formatted at 2026-08-08 22:36:09 on dist-test-slave-266d"
server_key: "dc5efc9559f3d4f76b6616c4b678ac72"
server_key_iv: "3c09a423abe0f40441750fe59bf678e3"
server_key_version: "encryptionkey@0"
I20260808 22:36:09.159096 2180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/minicluster-data/master-1/wal/instance:
uuid: "5595372de9c648ec9277eac84bc4fc28"
format_stamp: "Formatted at 2026-08-08 22:36:09 on dist-test-slave-266d"
server_key: "dc5efc9559f3d4f76b6616c4b678ac72"
server_key_iv: "3c09a423abe0f40441750fe59bf678e3"
server_key_version: "encryptionkey@0"
I20260808 22:36:09.162401 2180 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 22:36:09.164515 2202 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:36:09.165633 2180 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:36:09.165736 2180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/minicluster-data/master-1/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/minicluster-data/master-1/wal
uuid: "5595372de9c648ec9277eac84bc4fc28"
format_stamp: "Formatted at 2026-08-08 22:36:09 on dist-test-slave-266d"
server_key: "dc5efc9559f3d4f76b6616c4b678ac72"
server_key_iv: "3c09a423abe0f40441750fe59bf678e3"
server_key_version: "encryptionkey@0"
I20260808 22:36:09.165827 2180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:36:09.190640 2180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:36:09.193542 2180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:36:09.193704 2180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:36:09.200054 2180 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.189:33235
I20260808 22:36:09.200295 2282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.189:33235 every 8 connection(s)
I20260808 22:36:09.200910 2180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/minicluster-data/master-1/data/info.pb
I20260808 22:36:09.203434 2283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:36:09.206831 2283 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.87.190" port: 44685 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:36:09.209129 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 2180
I20260808 22:36:09.209273 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/minicluster-data/master-1/wal/instance
I20260808 22:36:09.209571 32094 external_mini_cluster.cc:1428] Setting key f674d6bf73d9fedd414c3cee9c528658
Aug 08 22:36:09 dist-test-slave-266d krb5kdc[2027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228569, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.189@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.87.188@KRBTEST.COM; defaulting to no policy
I20260808 22:36:09.219971 2172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:09.208431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43880 (local address 127.31.87.190:44685)
0808 22:36:09.208543 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:09.208546 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:09.209341 (+ 795us) server_negotiation.cc:407] Connection header received
0808 22:36:09.210408 (+ 1067us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:09.210411 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:09.210460 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:09.210529 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:09.211893 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:09.212674 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:09.213826 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:09.214202 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:09.214317 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:09.216789 (+ 2472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:36:09.216792 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:36:09.216794 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:36:09.216819 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:36:09.218100 (+ 1281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:09.218648 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:09.218651 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:09.218653 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:09.218732 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:09.219063 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:09.219065 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:09.219066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:09.219231 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:36:09.219300 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 22:36:09.219748 (+ 448us) server_negotiation.cc:299] Negotiation successful
0808 22:36:09.219852 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 22:36:09.220656 2285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:09.208705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:44685 (local address 127.0.0.1:43880)
0808 22:36:09.209173 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0808 22:36:09.209231 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 22:36:09.210199 (+ 968us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:36:09.210711 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:36:09.210719 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:36:09.211118 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:36:09.211755 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:36:09.211769 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:36:09.212806 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:36:09.212810 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:36:09.213704 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:36:09.213715 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:36:09.214349 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:36:09.214354 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:36:09.214506 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:09.214994 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:36:09.215013 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:36:09.216623 (+ 1610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:36:09.218229 (+ 1606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:36:09.218235 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:36:09.218249 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:36:09.218558 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:36:09.218816 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:36:09.218819 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:36:09.218821 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:36:09.218990 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:36:09.219305 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:36:09.219309 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:36:09.219519 (+ 210us) client_negotiation.cc:769] Sending connection context
0808 22:36:09.219684 (+ 165us) client_negotiation.cc:240] Negotiation successful
0808 22:36:09.219926 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":92,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.188@KRBTEST.COM" created.
I20260808 22:36:09.222074 2283 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.87.189" port: 33235 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:36:09.224613 2165 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.87.189" port: 33235 } attempt: 3
Aug 08 22:36:09 dist-test-slave-266d krb5kdc[2027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228569, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.189@KRBTEST.COM for kudu/127.31.87.189@KRBTEST.COM
I20260808 22:36:09.231330 2285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:09.222516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.189:33235 (local address 127.0.0.1:34738)
0808 22:36:09.222618 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0808 22:36:09.222632 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:36:09.222700 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:36:09.223270 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:36:09.223273 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:36:09.223435 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:36:09.223547 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:36:09.223553 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:36:09.224734 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:36:09.224737 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:36:09.225464 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:36:09.225476 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:36:09.226238 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:36:09.226243 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:36:09.226347 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:09.226818 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:36:09.226832 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:36:09.228288 (+ 1456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:36:09.230064 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:36:09.230066 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:36:09.230067 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:36:09.230237 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:36:09.230525 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:36:09.230528 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:36:09.230529 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:36:09.230598 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:36:09.230956 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:36:09.230958 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:36:09.231029 (+ 71us) client_negotiation.cc:769] Sending connection context
0808 22:36:09.231113 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 22:36:09.231209 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 22:36:09.231705 2292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:09.222641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34738 (local address 127.31.87.189:33235)
0808 22:36:09.222950 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:09.222963 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:09.222979 (+ 16us) server_negotiation.cc:407] Connection header received
0808 22:36:09.223010 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:09.223014 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:09.223144 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:09.223236 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:09.223655 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:09.224618 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:09.225584 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:09.226115 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:09.226226 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:09.228418 (+ 2192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:36:09.228428 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:36:09.228441 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:36:09.228465 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:36:09.229970 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:09.230346 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:09.230351 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:09.230352 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:09.230437 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:09.230662 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:09.230664 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:09.230666 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:09.230877 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:36:09.230993 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 22:36:09.231201 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 22:36:09.231312 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":147,"threads_started":1}
Aug 08 22:36:09 dist-test-slave-266d krb5kdc[2027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228568, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for kudu/127.31.87.189@KRBTEST.COM
I20260808 22:36:09.233374 2283 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37639 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:36:09.234035 2285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:09.233776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.188:37639 (local address 127.0.0.1:44712)
0808 22:36:09.233892 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0808 22:36:09.233968 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.188:37639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 22:36:09.234256 2206 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.87.188:37639: connect: Connection refused (error 111)
I20260808 22:36:09.235033 2285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:09.234832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.188:37639 (local address 127.0.0.1:44720)
0808 22:36:09.234925 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0808 22:36:09.234973 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.188:37639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 22:36:09.235178 2283 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.87.188:37639: Network error: Client connection negotiation failed: client connection to 127.31.87.188:37639: connect: Connection refused (error 111)
I20260808 22:36:09.235543 2173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:09.224985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.189:33235 (local address 127.0.0.1:34754)
0808 22:36:09.225181 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 22:36:09.225198 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 22:36:09.225285 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:36:09.225882 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:36:09.225885 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:36:09.226021 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:36:09.226523 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:36:09.226534 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:36:09.227462 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:36:09.227465 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:36:09.227966 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:36:09.227975 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:36:09.228456 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:36:09.228460 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:36:09.228569 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:09.229116 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:36:09.229133 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:36:09.232213 (+ 3080us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:36:09.233983 (+ 1770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:36:09.233986 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:36:09.233988 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:36:09.234214 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:36:09.234501 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:36:09.234504 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:36:09.234506 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:36:09.234610 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:36:09.235031 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:36:09.235035 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:36:09.235142 (+ 107us) client_negotiation.cc:769] Sending connection context
0808 22:36:09.235250 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 22:36:09.235384 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127}
I20260808 22:36:09.235569 2293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:09.225029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34754 (local address 127.31.87.189:33235)
0808 22:36:09.225699 (+ 670us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:09.225704 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:09.225713 (+ 9us) server_negotiation.cc:407] Connection header received
0808 22:36:09.225736 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:09.225738 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:09.225767 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:09.225842 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:09.226636 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:09.227357 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:09.228091 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:09.228320 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:09.228442 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:09.232422 (+ 3980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:36:09.232425 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:36:09.232427 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:36:09.232447 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:36:09.233881 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:09.234323 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:09.234325 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:09.234327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:09.234407 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:09.234698 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:09.234701 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:09.234703 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:09.234904 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:36:09.235053 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 22:36:09.235317 (+ 264us) server_negotiation.cc:299] Negotiation successful
0808 22:36:09.235415 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":610,"thread_start_us":119,"threads_started":1}
I20260808 22:36:09.236205 2165 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37639 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:36:09.236804 2173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:09.236603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.188:37639 (local address 127.0.0.1:44726)
0808 22:36:09.236704 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0808 22:36:09.236749 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.188:37639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.188.keytab.
Entry for principal kudu/127.31.87.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.188.keytab.
I20260808 22:36:09.237445 2173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:09.237216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.188:37639 (local address 127.0.0.1:44742)
0808 22:36:09.237352 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 22:36:09.237393 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.188:37639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260808 22:36:09.237561 2165 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.87.188:37639: Network error: Client connection negotiation failed: client connection to 127.31.87.188:37639: connect: Connection refused (error 111)
I20260808 22:36:09.237756 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.188:37639
--webserver_interface=127.31.87.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.188.keytab
--principal=kudu/127.31.87.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:43083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:44685,127.31.87.189:33235,127.31.87.188:37639
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 22:36:09.268690 2165 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37639 } attempt: 1
I20260808 22:36:09.269474 2173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:09.269147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.188:37639 (local address 127.0.0.1:44750)
0808 22:36:09.269305 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 22:36:09.269382 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.188:37639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260808 22:36:09.270205 2173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:09.269937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.188:37639 (local address 127.0.0.1:44762)
0808 22:36:09.270071 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 22:36:09.270130 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.188:37639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 22:36:09.270358 2165 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.87.188:37639: Network error: Client connection negotiation failed: client connection to 127.31.87.188:37639: connect: Connection refused (error 111)
I20260808 22:36:09.284395 2283 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37639 } attempt: 1
I20260808 22:36:09.284979 2285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:09.284752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.188:37639 (local address 127.0.0.1:44764)
0808 22:36:09.284867 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 22:36:09.284915 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.188:37639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 22:36:09.285719 2285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:09.285531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.188:37639 (local address 127.0.0.1:44772)
0808 22:36:09.285629 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0808 22:36:09.285670 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.188:37639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 22:36:09.285854 2283 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.87.188:37639: Network error: Client connection negotiation failed: client connection to 127.31.87.188:37639: connect: Connection refused (error 111)
I20260808 22:36:09.345515 2165 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37639 } attempt: 2
I20260808 22:36:09.346338 2173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:09.346021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.188:37639 (local address 127.0.0.1:44784)
0808 22:36:09.346179 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 22:36:09.346246 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.188:37639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260808 22:36:09.346940 2173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:09.346759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.188:37639 (local address 127.0.0.1:44794)
0808 22:36:09.346850 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0808 22:36:09.346892 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.188:37639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 22:36:09.347071 2165 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.87.188:37639: Network error: Client connection negotiation failed: client connection to 127.31.87.188:37639: connect: Connection refused (error 111)
W20260808 22:36:09.346923 2295 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 22:36:09.347195 2295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:36:09.347230 2295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:36:09.347265 2295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:36:09.351130 2295 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:36:09.351178 2295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:36:09.351197 2295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:36:09.351217 2295 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:36:09.351236 2295 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:36:09.353957 2283 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37639 } attempt: 2
I20260808 22:36:09.354604 2285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:09.354377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.188:37639 (local address 127.0.0.1:44796)
0808 22:36:09.354494 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0808 22:36:09.354541 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.188:37639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 22:36:09.355223 2285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:09.355038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.188:37639 (local address 127.0.0.1:44798)
0808 22:36:09.355131 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0808 22:36:09.355172 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.188:37639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 22:36:09.355330 2283 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.87.188:37639: Network error: Client connection negotiation failed: client connection to 127.31.87.188:37639: connect: Connection refused (error 111)
I20260808 22:36:09.356516 2295 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:43083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:44685,127.31.87.189:33235,127.31.87.188:37639
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.188.keytab
--principal=kudu/127.31.87.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.188:37639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.31.87.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:36:09.357457 2295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:36:09.358340 2295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:36:09.366190 2305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:36:09.366205 2304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:36:09.366380 2312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:36:09.366441 2295 server_base.cc:1034] running on GCE node
I20260808 22:36:09.367022 2295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:36:09.368052 2295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:36:09.369199 2295 hybrid_clock.cc:648] HybridClock initialized: now 1786228569369179 us; error 34 us; skew 500 ppm
Aug 08 22:36:09 dist-test-slave-266d krb5kdc[2027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228569, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:36:09.376041 2295 init.cc:377] Logged in from keytab as kudu/127.31.87.188@KRBTEST.COM (short username kudu)
I20260808 22:36:09.376899 2295 webserver.cc:460] Webserver started at http://127.31.87.188:40517/ using document root <none> and password file <none>
I20260808 22:36:09.377442 2295 fs_manager.cc:359] Metadata directory not provided
I20260808 22:36:09.377509 2295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:36:09.377713 2295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:36:09.379567 2295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/minicluster-data/master-2/data/instance:
uuid: "1654884cb174485e8f06332ca4f1ebab"
format_stamp: "Formatted at 2026-08-08 22:36:09 on dist-test-slave-266d"
server_key: "229139ea3ef9d778d37e2d805704ae76"
server_key_iv: "e0669db9c0c1fdd1be521f87fdd317ac"
server_key_version: "encryptionkey@0"
I20260808 22:36:09.380095 2295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/minicluster-data/master-2/wal/instance:
uuid: "1654884cb174485e8f06332ca4f1ebab"
format_stamp: "Formatted at 2026-08-08 22:36:09 on dist-test-slave-266d"
server_key: "229139ea3ef9d778d37e2d805704ae76"
server_key_iv: "e0669db9c0c1fdd1be521f87fdd317ac"
server_key_version: "encryptionkey@0"
I20260808 22:36:09.383512 2295 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260808 22:36:09.385684 2327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:36:09.386847 2295 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:36:09.386971 2295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/minicluster-data/master-2/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/minicluster-data/master-2/wal
uuid: "1654884cb174485e8f06332ca4f1ebab"
format_stamp: "Formatted at 2026-08-08 22:36:09 on dist-test-slave-266d"
server_key: "229139ea3ef9d778d37e2d805704ae76"
server_key_iv: "e0669db9c0c1fdd1be521f87fdd317ac"
server_key_version: "encryptionkey@0"
I20260808 22:36:09.387058 2295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:36:09.416790 2295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:36:09.419337 2295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:36:09.419497 2295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:36:09.425791 2295 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.188:37639
I20260808 22:36:09.426016 2400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.188:37639 every 8 connection(s)
I20260808 22:36:09.426648 2295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/minicluster-data/master-2/data/info.pb
I20260808 22:36:09.429224 2401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:36:09.432662 2401 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.87.190" port: 44685 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:36:09.434016 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 2295
I20260808 22:36:09.434141 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786228291013016-32094-0/minicluster-data/master-2/wal/instance
I20260808 22:36:09.434437 32094 external_mini_cluster.cc:1428] Setting key 08bb13c014d3fd52f95407aa7d2e845c
Aug 08 22:36:09 dist-test-slave-266d krb5kdc[2027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228569, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.188@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
Aug 08 22:36:09 dist-test-slave-266d krb5kdc[2027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.87.188@KRBTEST.COM
I20260808 22:36:09.445761 2172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:09.434393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43886 (local address 127.31.87.190:44685)
0808 22:36:09.434511 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:09.434516 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:09.435457 (+ 941us) server_negotiation.cc:407] Connection header received
0808 22:36:09.436408 (+ 951us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:09.436412 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:09.436463 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:09.436543 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:09.437537 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:09.438318 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:09.439558 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:09.439867 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:09.439971 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:09.442344 (+ 2373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:36:09.442348 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:36:09.442351 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:36:09.442380 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:36:09.443721 (+ 1341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:09.444265 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:09.444267 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:09.444269 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:09.444330 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:09.444651 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:09.444655 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:09.444656 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:09.444881 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:36:09.444972 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 22:36:09.445468 (+ 496us) server_negotiation.cc:299] Negotiation successful
0808 22:36:09.445582 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 22:36:09.446209 2165 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37639 } attempt: 3
I20260808 22:36:09.446472 2283 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37639 } attempt: 3
I20260808 22:36:09.446607 2403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:09.434840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:44685 (local address 127.0.0.1:43886)
0808 22:36:09.435304 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0808 22:36:09.435356 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 22:36:09.436192 (+ 836us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:36:09.436597 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:36:09.436602 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:36:09.437016 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:36:09.437390 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:36:09.437409 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:36:09.438411 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:36:09.438413 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:36:09.439436 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:36:09.439447 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:36:09.439998 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:36:09.440001 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:36:09.440139 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:09.440631 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:36:09.440652 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:36:09.442153 (+ 1501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:36:09.443825 (+ 1672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:36:09.443832 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:36:09.443841 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:36:09.444184 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:36:09.444398 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:36:09.444401 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:36:09.444403 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:36:09.444552 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:36:09.444966 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:36:09.444971 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:36:09.445231 (+ 260us) client_negotiation.cc:769] Sending connection context
0808 22:36:09.445422 (+ 191us) client_negotiation.cc:240] Negotiation successful
0808 22:36:09.445692 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":123,"threads_started":1}
I20260808 22:36:09.446607 2406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:09.435063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44808 (local address 127.31.87.188:37639)
0808 22:36:09.435429 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:09.435440 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:09.435527 (+ 87us) server_negotiation.cc:407] Connection header received
0808 22:36:09.436197 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:09.436211 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:09.436412 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:09.436587 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:09.437433 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:09.438593 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:09.439662 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:09.440153 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:09.440227 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:09.442738 (+ 2511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:36:09.442745 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:36:09.442757 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:36:09.442781 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:36:09.444380 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:09.444767 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:09.444773 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:09.444775 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:09.444857 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:09.445133 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:09.445136 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:09.445137 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:09.445386 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:36:09.445571 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0808 22:36:09.445745 (+ 174us) server_negotiation.cc:299] Negotiation successful
0808 22:36:09.445861 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":115,"threads_started":1}
I20260808 22:36:09.448287 2401 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.87.189" port: 33235 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 22:36:09 dist-test-slave-266d krb5kdc[2027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228568, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for kudu/127.31.87.188@KRBTEST.COM
Aug 08 22:36:09 dist-test-slave-266d krb5kdc[2027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228569, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.189@KRBTEST.COM for kudu/127.31.87.188@KRBTEST.COM
Aug 08 22:36:09 dist-test-slave-266d krb5kdc[2027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
Aug 08 22:36:09 dist-test-slave-266d krb5kdc[2027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228569, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.188@KRBTEST.COM for kudu/127.31.87.189@KRBTEST.COM
I20260808 22:36:09.459429 2173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:09.446540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.188:37639 (local address 127.0.0.1:44810)
0808 22:36:09.446792 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0808 22:36:09.446805 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:36:09.446870 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:36:09.447537 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:36:09.447540 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:36:09.447699 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:36:09.447802 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:36:09.447809 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:36:09.448785 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:36:09.448788 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:36:09.449496 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:36:09.449504 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:36:09.450223 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:36:09.450227 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:36:09.450306 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:09.450746 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:36:09.450760 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:36:09.452663 (+ 1903us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:36:09.457935 (+ 5272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:36:09.457939 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:36:09.457940 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:36:09.458169 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:36:09.458491 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:36:09.458494 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:36:09.458496 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:36:09.458585 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:36:09.459005 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:36:09.459008 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:36:09.459093 (+ 85us) client_negotiation.cc:769] Sending connection context
0808 22:36:09.459168 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 22:36:09.459283 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183}
I20260808 22:36:09.459575 2403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:09.448736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.189:33235 (local address 127.0.0.1:34760)
0808 22:36:09.449610 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0808 22:36:09.449620 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 22:36:09.449685 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:36:09.449953 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:36:09.449955 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:36:09.450101 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:36:09.450270 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:36:09.450278 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:36:09.451280 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:36:09.451284 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:36:09.452053 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:36:09.452064 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:36:09.452756 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:36:09.452760 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:36:09.452849 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:09.453221 (+ 372us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:36:09.453236 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:36:09.454695 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:36:09.457957 (+ 3262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:36:09.457960 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:36:09.457961 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:36:09.458272 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:36:09.458557 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:36:09.458560 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:36:09.458562 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:36:09.458659 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:36:09.459143 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:36:09.459146 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:36:09.459237 (+ 91us) client_negotiation.cc:769] Sending connection context
0808 22:36:09.459327 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 22:36:09.459432 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":822,"mutex_wait_us":47,"spinlock_wait_cycles":2816}
I20260808 22:36:09.459820 2172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:09.448954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43888 (local address 127.31.87.190:44685)
0808 22:36:09.449042 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:09.449045 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:09.449056 (+ 11us) server_negotiation.cc:407] Connection header received
0808 22:36:09.449113 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:09.449116 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:09.449152 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:09.449225 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:09.449935 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:09.450595 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:09.451619 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:09.451908 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:09.451995 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:09.454883 (+ 2888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:36:09.454887 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:36:09.454890 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:36:09.454914 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:36:09.457731 (+ 2817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:09.458258 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:09.458262 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:09.458263 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:09.458349 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:09.458631 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:09.458634 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:09.458636 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:09.458825 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:36:09.459029 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0808 22:36:09.459592 (+ 563us) server_negotiation.cc:299] Negotiation successful
0808 22:36:09.459692 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 22:36:09.460084 2285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:09.446839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.188:37639 (local address 127.0.0.1:44814)
0808 22:36:09.447008 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 22:36:09.447021 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:36:09.447093 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:36:09.448117 (+ 1024us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:36:09.448120 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:36:09.448294 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:36:09.448398 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:36:09.448405 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:36:09.450351 (+ 1946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:36:09.450354 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:36:09.451125 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:36:09.451136 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:36:09.451712 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:36:09.451716 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:36:09.451796 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:09.452216 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:36:09.452230 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:36:09.453621 (+ 1391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:36:09.458509 (+ 4888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:36:09.458511 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:36:09.458513 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:36:09.458729 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:36:09.459005 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:36:09.459008 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:36:09.459010 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:36:09.459092 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:36:09.459632 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:36:09.459635 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:36:09.459731 (+ 96us) client_negotiation.cc:769] Sending connection context
0808 22:36:09.459808 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 22:36:09.459934 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102}
I20260808 22:36:09.460402 2407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:09.447005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44810 (local address 127.31.87.188:37639)
0808 22:36:09.447303 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:09.447311 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:09.447324 (+ 13us) server_negotiation.cc:407] Connection header received
0808 22:36:09.447353 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:09.447356 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:09.447392 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:09.447906 (+ 514us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:09.447986 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:09.448655 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:09.449795 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:09.450099 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:09.452784 (+ 2685us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:09.452874 (+ 90us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:36:09.452877 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:36:09.452879 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:36:09.452897 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:36:09.457811 (+ 4914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:09.458289 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:09.458293 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:09.458294 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:09.458385 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:09.458697 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:09.458700 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:09.458702 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:09.458900 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:36:09.460076 (+ 1176us) server_negotiation.cc:1035] Waiting for connection context
0808 22:36:09.460150 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 22:36:09.460252 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":127,"threads_started":1}
I20260808 22:36:09.460777 2293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:09.448955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34760 (local address 127.31.87.189:33235)
0808 22:36:09.449073 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:09.449076 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:09.449680 (+ 604us) server_negotiation.cc:407] Connection header received
0808 22:36:09.449752 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:09.449755 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:09.449798 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:09.449860 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:09.450405 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:09.451131 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:09.452184 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:09.452550 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:09.452624 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:09.454819 (+ 2195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:36:09.454822 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:36:09.454825 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:36:09.454848 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:36:09.457833 (+ 2985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:09.458373 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:09.458377 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:09.458378 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:09.458459 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:09.458760 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:09.458763 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:09.458765 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:09.459013 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:36:09.459100 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 22:36:09.459828 (+ 728us) server_negotiation.cc:299] Negotiation successful
0808 22:36:09.460633 (+ 805us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":683,"server-negotiator.queue_time_us":57}
I20260808 22:36:09.461447 2401 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37639 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:36:09.461737 2408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:09.447392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44814 (local address 127.31.87.188:37639)
0808 22:36:09.447881 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:09.447888 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:09.447900 (+ 12us) server_negotiation.cc:407] Connection header received
0808 22:36:09.447927 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:09.447930 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:09.447965 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:09.448506 (+ 541us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:09.448606 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:09.450234 (+ 1628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:09.451263 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:09.451598 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:09.451996 (+ 398us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:09.453767 (+ 1771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:36:09.453771 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:36:09.453774 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:36:09.453801 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:36:09.458406 (+ 4605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:09.458838 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:09.458841 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:09.458843 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:09.458908 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:09.459202 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:09.459204 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:09.459206 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:09.459405 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:36:09.461119 (+ 1714us) server_negotiation.cc:1035] Waiting for connection context
0808 22:36:09.461190 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 22:36:09.461588 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":429,"spinlock_wait_cycles":1408,"thread_start_us":89,"threads_started":1}
I20260808 22:36:09.463865 2165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23ca781363d048a5a98b2e5db8a9f370: Bootstrap starting.
Aug 08 22:36:09 dist-test-slave-266d krb5kdc[2027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.87.189@KRBTEST.COM
Aug 08 22:36:09 dist-test-slave-266d krb5kdc[2027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228569, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.188@KRBTEST.COM for kudu/127.31.87.188@KRBTEST.COM
I20260808 22:36:09.466668 2165 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 23ca781363d048a5a98b2e5db8a9f370: Neither blocks nor log segments found. Creating new log.
I20260808 22:36:09.469410 2293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:09.461216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34772 (local address 127.31.87.189:33235)
0808 22:36:09.461312 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:09.461316 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:09.461328 (+ 12us) server_negotiation.cc:407] Connection header received
0808 22:36:09.461359 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:09.461361 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:09.461398 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:09.461465 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:09.462122 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:09.462728 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:09.463578 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:09.463917 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:09.463990 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:09.466205 (+ 2215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:36:09.466209 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:36:09.466211 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:36:09.466235 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:36:09.467587 (+ 1352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:09.467945 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:09.467949 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:09.467951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:09.468036 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:09.468326 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:09.468329 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:09.468331 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:09.468523 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:36:09.468704 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0808 22:36:09.468896 (+ 192us) server_negotiation.cc:299] Negotiation successful
0808 22:36:09.468997 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 22:36:09.469498 2283 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5595372de9c648ec9277eac84bc4fc28: Bootstrap starting.
I20260808 22:36:09.470474 2403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:09.461821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.188:37639 (local address 127.0.0.1:44816)
0808 22:36:09.461911 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0808 22:36:09.461923 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:36:09.461986 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:36:09.462496 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:36:09.462499 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:36:09.462661 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:36:09.462742 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:36:09.462748 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:36:09.463627 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:36:09.463630 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:36:09.464280 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:36:09.464291 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:36:09.464903 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:36:09.464906 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:36:09.464975 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:09.465431 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:36:09.465443 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:36:09.466814 (+ 1371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:36:09.468911 (+ 2097us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:36:09.468913 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:36:09.468915 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:36:09.469162 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:36:09.469538 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:36:09.469541 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:36:09.469542 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:36:09.469625 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:36:09.469973 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:36:09.469976 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:36:09.470050 (+ 74us) client_negotiation.cc:769] Sending connection context
0808 22:36:09.470130 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 22:36:09.470247 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 22:36:09.470487 2408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:09.462131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44816 (local address 127.31.87.188:37639)
0808 22:36:09.462305 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:09.462309 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:09.462321 (+ 12us) server_negotiation.cc:407] Connection header received
0808 22:36:09.462350 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:09.462352 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:09.462385 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:09.462455 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:09.462858 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:09.463506 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:09.464394 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:09.464724 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:09.464813 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:09.466982 (+ 2169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:36:09.466986 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:36:09.466988 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:36:09.467011 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:36:09.468813 (+ 1802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:09.469289 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:09.469293 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:09.469295 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:09.469369 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:09.469732 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:09.469735 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:09.469737 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:09.469887 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:36:09.469962 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 22:36:09.470182 (+ 220us) server_negotiation.cc:299] Negotiation successful
0808 22:36:09.470301 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":120}
I20260808 22:36:09.472234 2283 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5595372de9c648ec9277eac84bc4fc28: Neither blocks nor log segments found. Creating new log.
I20260808 22:36:09.473162 2283 log.cc:824] T 00000000000000000000000000000000 P 5595372de9c648ec9277eac84bc4fc28: Log is configured to *not* fsync() on all Append() calls
I20260808 22:36:09.473687 2401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1654884cb174485e8f06332ca4f1ebab: Bootstrap starting.
I20260808 22:36:09.474272 2165 log.cc:824] T 00000000000000000000000000000000 P 23ca781363d048a5a98b2e5db8a9f370: Log is configured to *not* fsync() on all Append() calls
I20260808 22:36:09.474998 32094 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:36:09.475922 2401 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1654884cb174485e8f06332ca4f1ebab: Neither blocks nor log segments found. Creating new log.
I20260808 22:36:09.476714 2401 log.cc:824] T 00000000000000000000000000000000 P 1654884cb174485e8f06332ca4f1ebab: Log is configured to *not* fsync() on all Append() calls
I20260808 22:36:09.476792 2165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23ca781363d048a5a98b2e5db8a9f370: No bootstrap required, opened a new log
I20260808 22:36:09.478535 2401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1654884cb174485e8f06332ca4f1ebab: No bootstrap required, opened a new log
I20260808 22:36:09.480986 2165 raft_consensus.cc:348] T 00000000000000000000000000000000 P 23ca781363d048a5a98b2e5db8a9f370 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23ca781363d048a5a98b2e5db8a9f370" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 44685 } } peers { permanent_uuid: "5595372de9c648ec9277eac84bc4fc28" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 33235 } } peers { permanent_uuid: "1654884cb174485e8f06332ca4f1ebab" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37639 } }
I20260808 22:36:09.481587 2165 raft_consensus.cc:729] T 00000000000000000000000000000000 P 23ca781363d048a5a98b2e5db8a9f370 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23ca781363d048a5a98b2e5db8a9f370, State: Initialized, Role: FOLLOWER
I20260808 22:36:09.481545 2125 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "23ca781363d048a5a98b2e5db8a9f370"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43888
I20260808 22:36:09.481907 2165 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23ca781363d048a5a98b2e5db8a9f370 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23ca781363d048a5a98b2e5db8a9f370" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 44685 } } peers { permanent_uuid: "5595372de9c648ec9277eac84bc4fc28" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 33235 } } peers { permanent_uuid: "1654884cb174485e8f06332ca4f1ebab" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37639 } }
I20260808 22:36:09.482357 2401 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1654884cb174485e8f06332ca4f1ebab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23ca781363d048a5a98b2e5db8a9f370" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 44685 } } peers { permanent_uuid: "5595372de9c648ec9277eac84bc4fc28" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 33235 } } peers { permanent_uuid: "1654884cb174485e8f06332ca4f1ebab" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37639 } }
I20260808 22:36:09.482620 2401 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1654884cb174485e8f06332ca4f1ebab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1654884cb174485e8f06332ca4f1ebab, State: Initialized, Role: FOLLOWER
I20260808 22:36:09.482965 2401 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1654884cb174485e8f06332ca4f1ebab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23ca781363d048a5a98b2e5db8a9f370" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 44685 } } peers { permanent_uuid: "5595372de9c648ec9277eac84bc4fc28" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 33235 } } peers { permanent_uuid: "1654884cb174485e8f06332ca4f1ebab" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37639 } }
I20260808 22:36:09.484223 2401 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1654884cb174485e8f06332ca4f1ebab [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:36:09.484215 2413 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23ca781363d048a5a98b2e5db8a9f370 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23ca781363d048a5a98b2e5db8a9f370" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 44685 } } peers { permanent_uuid: "5595372de9c648ec9277eac84bc4fc28" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 33235 } } peers { permanent_uuid: "1654884cb174485e8f06332ca4f1ebab" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37639 } } }
I20260808 22:36:09.484315 2125 raft_consensus.cc:482] T 00000000000000000000000000000000 P 23ca781363d048a5a98b2e5db8a9f370 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 22:36:09.484377 2413 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23ca781363d048a5a98b2e5db8a9f370 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:36:09.484448 2125 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 23ca781363d048a5a98b2e5db8a9f370 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:36:09.485612 2125 raft_consensus.cc:504] T 00000000000000000000000000000000 P 23ca781363d048a5a98b2e5db8a9f370 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23ca781363d048a5a98b2e5db8a9f370" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 44685 } } peers { permanent_uuid: "5595372de9c648ec9277eac84bc4fc28" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 33235 } } peers { permanent_uuid: "1654884cb174485e8f06332ca4f1ebab" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37639 } }
I20260808 22:36:09.486378 2125 leader_election.cc:288] T 00000000000000000000000000000000 P 23ca781363d048a5a98b2e5db8a9f370 [CANDIDATE]: Term 1 election: Requested vote from peers 5595372de9c648ec9277eac84bc4fc28 (127.31.87.189:33235), 1654884cb174485e8f06332ca4f1ebab (127.31.87.188:37639)
I20260808 22:36:09.487548 2283 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5595372de9c648ec9277eac84bc4fc28: No bootstrap required, opened a new log
I20260808 22:36:09.488500 2366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "23ca781363d048a5a98b2e5db8a9f370" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "1654884cb174485e8f06332ca4f1ebab"
I20260808 22:36:09.488696 2366 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1654884cb174485e8f06332ca4f1ebab [term 0 FOLLOWER]: Advancing to term 1
W20260808 22:36:09.489060 2433 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1654884cb174485e8f06332ca4f1ebab: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:36:09.489147 2433 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:36:09.489894 2366 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1654884cb174485e8f06332ca4f1ebab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23ca781363d048a5a98b2e5db8a9f370 in term 1.
I20260808 22:36:09.490525 2086 leader_election.cc:302] T 00000000000000000000000000000000 P 23ca781363d048a5a98b2e5db8a9f370 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1654884cb174485e8f06332ca4f1ebab, 23ca781363d048a5a98b2e5db8a9f370; no voters:
I20260808 22:36:09.490793 2413 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 23ca781363d048a5a98b2e5db8a9f370 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:36:09.490983 2413 raft_consensus.cc:686] T 00000000000000000000000000000000 P 23ca781363d048a5a98b2e5db8a9f370 [term 1 LEADER]: Becoming Leader. State: Replica: 23ca781363d048a5a98b2e5db8a9f370, State: Running, Role: LEADER
I20260808 22:36:09.491281 2413 consensus_queue.cc:237] T 00000000000000000000000000000000 P 23ca781363d048a5a98b2e5db8a9f370 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23ca781363d048a5a98b2e5db8a9f370" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 44685 } } peers { permanent_uuid: "5595372de9c648ec9277eac84bc4fc28" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 33235 } } peers { permanent_uuid: "1654884cb174485e8f06332ca4f1ebab" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37639 } }
I20260808 22:36:09.488433 2295 master_runner.cc:186] Error getting master registration for 127.31.87.190:44685: OK, instance_id { permanent_uuid: "23ca781363d048a5a98b2e5db8a9f370" instance_seqno: 1786228568986051 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 22:36:09.491524 2295 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 22:36:09.484231 2165 sys_catalog.cc:564] T 00000000000000000000000000000000 P 23ca781363d048a5a98b2e5db8a9f370 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:36:09.491722 2283 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5595372de9c648ec9277eac84bc4fc28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23ca781363d048a5a98b2e5db8a9f370" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 44685 } } peers { permanent_uuid: "5595372de9c648ec9277eac84bc4fc28" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 33235 } } peers { permanent_uuid: "1654884cb174485e8f06332ca4f1ebab" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37639 } }
I20260808 22:36:09.491916 2283 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5595372de9c648ec9277eac84bc4fc28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5595372de9c648ec9277eac84bc4fc28, State: Initialized, Role: FOLLOWER
I20260808 22:36:09.492187 2283 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5595372de9c648ec9277eac84bc4fc28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23ca781363d048a5a98b2e5db8a9f370" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 44685 } } peers { permanent_uuid: "5595372de9c648ec9277eac84bc4fc28" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 33235 } } peers { permanent_uuid: "1654884cb174485e8f06332ca4f1ebab" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37639 } }
I20260808 22:36:09.492964 2413 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23ca781363d048a5a98b2e5db8a9f370 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23ca781363d048a5a98b2e5db8a9f370. Latest consensus state: current_term: 1 leader_uuid: "23ca781363d048a5a98b2e5db8a9f370" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23ca781363d048a5a98b2e5db8a9f370" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 44685 } } peers { permanent_uuid: "5595372de9c648ec9277eac84bc4fc28" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 33235 } } peers { permanent_uuid: "1654884cb174485e8f06332ca4f1ebab" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37639 } } }
I20260808 22:36:09.493063 2413 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23ca781363d048a5a98b2e5db8a9f370 [sys.catalog]: This master's current role is: LEADER
I20260808 22:36:09.493363 2283 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5595372de9c648ec9277eac84bc4fc28 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:36:09.493412 2243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "23ca781363d048a5a98b2e5db8a9f370" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "5595372de9c648ec9277eac84bc4fc28"
I20260808 22:36:09.493634 2243 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5595372de9c648ec9277eac84bc4fc28 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:36:09.494850 2243 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5595372de9c648ec9277eac84bc4fc28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23ca781363d048a5a98b2e5db8a9f370 in term 1.
I20260808 22:36:09.495352 2462 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:36:09.498687 2462 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:36:09.496932 2180 master_runner.cc:186] Error getting master registration for 127.31.87.190:44685: OK, instance_id { permanent_uuid: "23ca781363d048a5a98b2e5db8a9f370" instance_seqno: 1786228568986051 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 22:36:09.507052 2180 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 22:36:09.509919 2366 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1654884cb174485e8f06332ca4f1ebab [term 1 FOLLOWER]: Refusing update from remote peer 23ca781363d048a5a98b2e5db8a9f370: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260808 22:36:09.510309 2478 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5595372de9c648ec9277eac84bc4fc28: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:36:09.510377 2478 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:36:09.510524 2413 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 23ca781363d048a5a98b2e5db8a9f370 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1654884cb174485e8f06332ca4f1ebab" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:36:09.510824 2451 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5595372de9c648ec9277eac84bc4fc28 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23ca781363d048a5a98b2e5db8a9f370" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 44685 } } peers { permanent_uuid: "5595372de9c648ec9277eac84bc4fc28" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 33235 } } peers { permanent_uuid: "1654884cb174485e8f06332ca4f1ebab" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37639 } } }
I20260808 22:36:09.510933 2451 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5595372de9c648ec9277eac84bc4fc28 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:36:09.514263 2415 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1654884cb174485e8f06332ca4f1ebab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "23ca781363d048a5a98b2e5db8a9f370" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23ca781363d048a5a98b2e5db8a9f370" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 44685 } } peers { permanent_uuid: "5595372de9c648ec9277eac84bc4fc28" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 33235 } } peers { permanent_uuid: "1654884cb174485e8f06332ca4f1ebab" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37639 } } }
I20260808 22:36:09.514382 2415 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1654884cb174485e8f06332ca4f1ebab [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:36:09.514453 2415 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1654884cb174485e8f06332ca4f1ebab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23ca781363d048a5a98b2e5db8a9f370. Latest consensus state: current_term: 1 leader_uuid: "23ca781363d048a5a98b2e5db8a9f370" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23ca781363d048a5a98b2e5db8a9f370" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 44685 } } peers { permanent_uuid: "5595372de9c648ec9277eac84bc4fc28" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 33235 } } peers { permanent_uuid: "1654884cb174485e8f06332ca4f1ebab" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37639 } } }
I20260808 22:36:09.514509 2415 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1654884cb174485e8f06332ca4f1ebab [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:36:09.515215 2243 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5595372de9c648ec9277eac84bc4fc28 [term 1 FOLLOWER]: Refusing update from remote peer 23ca781363d048a5a98b2e5db8a9f370: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:36:09.515762 2413 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 23ca781363d048a5a98b2e5db8a9f370 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5595372de9c648ec9277eac84bc4fc28" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 33235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:36:09.516484 2172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:09.489511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43900 (local address 127.31.87.190:44685)
0808 22:36:09.490166 (+ 655us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:09.490170 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:09.507284 (+ 17114us) server_negotiation.cc:407] Connection header received
0808 22:36:09.507420 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:09.507423 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:09.507471 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:09.507538 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:09.508499 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:09.509207 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:09.510362 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:09.510675 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:09.510750 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:09.512715 (+ 1965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:36:09.512719 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:36:09.512721 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:36:09.512747 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:36:09.514396 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:09.514962 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:09.514966 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:09.514968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:09.515054 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:09.515441 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:09.515445 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:09.515446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:09.515663 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:36:09.515761 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 22:36:09.516205 (+ 444us) server_negotiation.cc:299] Negotiation successful
0808 22:36:09.516341 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566}
I20260808 22:36:09.520392 2451 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5595372de9c648ec9277eac84bc4fc28 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23ca781363d048a5a98b2e5db8a9f370. Latest consensus state: current_term: 1 leader_uuid: "23ca781363d048a5a98b2e5db8a9f370" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23ca781363d048a5a98b2e5db8a9f370" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 44685 } } peers { permanent_uuid: "5595372de9c648ec9277eac84bc4fc28" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 33235 } } peers { permanent_uuid: "1654884cb174485e8f06332ca4f1ebab" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37639 } } }
I20260808 22:36:09.520527 2451 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5595372de9c648ec9277eac84bc4fc28 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:36:09.521659 2408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:09.490212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44824 (local address 127.31.87.188:37639)
0808 22:36:09.490312 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:09.490316 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:09.511212 (+ 20896us) server_negotiation.cc:407] Connection header received
0808 22:36:09.511366 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:09.511369 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:09.511412 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:09.511458 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:09.512302 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:09.513032 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:09.514624 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:09.514956 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:09.515024 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:09.518278 (+ 3254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:36:09.518281 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:36:09.518284 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:36:09.518306 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:36:09.519699 (+ 1393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:09.520192 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:09.520195 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:09.520197 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:09.520283 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:09.520685 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:09.520689 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:09.520691 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:09.520897 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:36:09.521008 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 22:36:09.521396 (+ 388us) server_negotiation.cc:299] Negotiation successful
0808 22:36:09.521512 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 22:36:09.524166 2413 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23ca781363d048a5a98b2e5db8a9f370 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "23ca781363d048a5a98b2e5db8a9f370" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23ca781363d048a5a98b2e5db8a9f370" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 44685 } } peers { permanent_uuid: "5595372de9c648ec9277eac84bc4fc28" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 33235 } } peers { permanent_uuid: "1654884cb174485e8f06332ca4f1ebab" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37639 } } }
I20260808 22:36:09.524283 2413 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23ca781363d048a5a98b2e5db8a9f370 [sys.catalog]: This master's current role is: LEADER
I20260808 22:36:09.524461 2462 catalog_manager.cc:1347] Generated new cluster ID: 75e0ae6c790c4b4c9ab799e19a01ae9c
I20260808 22:36:09.524524 2462 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:36:09.525461 2451 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5595372de9c648ec9277eac84bc4fc28 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "23ca781363d048a5a98b2e5db8a9f370" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23ca781363d048a5a98b2e5db8a9f370" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 44685 } } peers { permanent_uuid: "5595372de9c648ec9277eac84bc4fc28" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 33235 } } peers { permanent_uuid: "1654884cb174485e8f06332ca4f1ebab" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37639 } } }
I20260808 22:36:09.525553 2451 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5595372de9c648ec9277eac84bc4fc28 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:36:09.529351 2293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:09.489841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34782 (local address 127.31.87.189:33235)
0808 22:36:09.489943 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:09.489947 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:09.507599 (+ 17652us) server_negotiation.cc:407] Connection header received
0808 22:36:09.507698 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:09.507701 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:09.507745 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:09.507815 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:09.518006 (+ 10191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:09.518976 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:09.519995 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:09.520320 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:09.520394 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:09.526272 (+ 5878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:36:09.526276 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:36:09.526279 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:36:09.526301 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:36:09.527767 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:09.528223 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:09.528226 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:09.528228 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:09.528297 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:09.528545 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:09.528548 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:09.528550 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:09.528732 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:36:09.528834 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 22:36:09.529073 (+ 239us) server_negotiation.cc:299] Negotiation successful
0808 22:36:09.529207 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 22:36:09.535336 2413 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23ca781363d048a5a98b2e5db8a9f370 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "23ca781363d048a5a98b2e5db8a9f370" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23ca781363d048a5a98b2e5db8a9f370" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 44685 } } peers { permanent_uuid: "5595372de9c648ec9277eac84bc4fc28" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 33235 } } peers { permanent_uuid: "1654884cb174485e8f06332ca4f1ebab" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37639 } } }
I20260808 22:36:09.535459 2413 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23ca781363d048a5a98b2e5db8a9f370 [sys.catalog]: This master's current role is: LEADER
I20260808 22:36:09.538866 2491 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1654884cb174485e8f06332ca4f1ebab [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "23ca781363d048a5a98b2e5db8a9f370" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23ca781363d048a5a98b2e5db8a9f370" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 44685 } } peers { permanent_uuid: "5595372de9c648ec9277eac84bc4fc28" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 33235 } } peers { permanent_uuid: "1654884cb174485e8f06332ca4f1ebab" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37639 } } }
I20260808 22:36:09.538959 2491 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1654884cb174485e8f06332ca4f1ebab [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:36:09.539161 2492 mvcc.cc:205] Tried to move back new op lower bound from 7316392220705722368 to 7316392220639657984. Current Snapshot: MvccSnapshot[applied={T|T < 7316392220705722368}]
I20260808 22:36:09.540256 2494 mvcc.cc:205] Tried to move back new op lower bound from 7316392220705722368 to 7316392220639657984. Current Snapshot: MvccSnapshot[applied={T|T < 7316392220705722368}]
I20260808 22:36:09.549630 2462 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:36:09.553153 2462 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:36:09.581681 2462 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 23ca781363d048a5a98b2e5db8a9f370: Generated new TSK 0
I20260808 22:36:09.582513 2462 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 22:36:09.627966 2172 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:36:09.624845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43902 (local address 127.31.87.190:44685)
0808 22:36:09.624996 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:09.625002 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:09.626836 (+ 1834us) server_negotiation.cc:407] Connection header received
0808 22:36:09.627026 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:09.627030 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:09.627092 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:09.627144 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:09.627822 (+ 678us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43902: BlockingRecv error: recv got EOF from 127.0.0.1:43902 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260808 22:36:09.629346 2293 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:36:09.625288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34794 (local address 127.31.87.189:33235)
0808 22:36:09.625400 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:09.625404 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:09.628447 (+ 3043us) server_negotiation.cc:407] Connection header received
0808 22:36:09.628619 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:09.628623 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:09.628670 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:09.628718 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:09.629231 (+ 513us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34794: BlockingRecv error: recv got EOF from 127.0.0.1:34794 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260808 22:36:09.630635 2408 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:36:09.625707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44832 (local address 127.31.87.188:37639)
0808 22:36:09.625815 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:09.625819 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:09.629796 (+ 3977us) server_negotiation.cc:407] Connection header received
0808 22:36:09.629959 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:09.629962 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:09.630007 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:09.630055 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:09.630523 (+ 468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44832: BlockingRecv error: recv got EOF from 127.0.0.1:44832 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260808 22:36:09.634892 2293 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:36:09.633461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34810 (local address 127.31.87.189:33235)
0808 22:36:09.633557 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:09.633561 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:09.634009 (+ 448us) server_negotiation.cc:407] Connection header received
0808 22:36:09.634225 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:09.634228 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:09.634295 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:09.634343 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:09.634804 (+ 461us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34810: BlockingRecv error: recv got EOF from 127.0.0.1:34810 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260808 22:36:09.639721 2293 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:36:09.638347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34824 (local address 127.31.87.189:33235)
0808 22:36:09.638436 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:09.638440 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:09.638931 (+ 491us) server_negotiation.cc:407] Connection header received
0808 22:36:09.639109 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:09.639112 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:09.639153 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:09.639197 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:09.639639 (+ 442us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34824: BlockingRecv error: recv got EOF from 127.0.0.1:34824 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260808 22:36:09.645032 2293 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:36:09.643685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34836 (local address 127.31.87.189:33235)
0808 22:36:09.643772 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:09.643775 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:09.644263 (+ 488us) server_negotiation.cc:407] Connection header received
0808 22:36:09.644423 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:09.644426 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:09.644468 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:09.644513 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:09.644949 (+ 436us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34836: BlockingRecv error: recv got EOF from 127.0.0.1:34836 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260808 22:36:09.652311 2293 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:36:09.650979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34848 (local address 127.31.87.189:33235)
0808 22:36:09.651099 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:09.651103 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:09.651557 (+ 454us) server_negotiation.cc:407] Connection header received
0808 22:36:09.651714 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:09.651717 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:09.651757 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:09.651800 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:09.652230 (+ 430us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34848: BlockingRecv error: recv got EOF from 127.0.0.1:34848 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260808 22:36:09.663913 2293 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:36:09.662376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34864 (local address 127.31.87.189:33235)
0808 22:36:09.662475 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:09.662479 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:09.663008 (+ 529us) server_negotiation.cc:407] Connection header received
0808 22:36:09.663215 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:09.663218 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:09.663265 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:09.663314 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:09.663815 (+ 501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34864: BlockingRecv error: recv got EOF from 127.0.0.1:34864 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260808 22:36:09.684163 2293 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:36:09.682321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34874 (local address 127.31.87.189:33235)
0808 22:36:09.682441 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:09.682445 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:09.683064 (+ 619us) server_negotiation.cc:407] Connection header received
0808 22:36:09.683304 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:09.683307 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:09.683363 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:09.683418 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:09.684026 (+ 608us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34874: BlockingRecv error: recv got EOF from 127.0.0.1:34874 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260808 22:36:09.719308 2293 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:36:09.718290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34882 (local address 127.31.87.189:33235)
0808 22:36:09.718426 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:09.718430 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:09.718554 (+ 124us) server_negotiation.cc:407] Connection header received
0808 22:36:09.718668 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:09.718671 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:09.718725 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:09.718822 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:09.719190 (+ 368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34882: BlockingRecv error: recv got EOF from 127.0.0.1:34882 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260808 22:36:09.786347 2293 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:36:09.785221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42614 (local address 127.31.87.189:33235)
0808 22:36:09.785377 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:09.785382 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:09.785522 (+ 140us) server_negotiation.cc:407] Connection header received
0808 22:36:09.785716 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:09.785720 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:09.785781 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:09.785852 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:09.786227 (+ 375us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42614: BlockingRecv error: recv got EOF from 127.0.0.1:42614 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260808 22:36:09.917510 2293 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:36:09.916295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42616 (local address 127.31.87.189:33235)
0808 22:36:09.916565 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:09.916571 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:09.916699 (+ 128us) server_negotiation.cc:407] Connection header received
0808 22:36:09.916816 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:09.916818 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:09.916871 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:09.916962 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:09.917376 (+ 414us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42616: BlockingRecv error: recv got EOF from 127.0.0.1:42616 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
W20260808 22:36:10.176752 2293 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:36:10.175663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42626 (local address 127.31.87.189:33235)
0808 22:36:10.175796 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:10.175801 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:10.175899 (+ 98us) server_negotiation.cc:407] Connection header received
0808 22:36:10.176055 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:10.176058 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:10.176106 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:10.176173 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:10.176651 (+ 478us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42626: BlockingRecv error: recv got EOF from 127.0.0.1:42626 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 22:36:10.490475 2433 catalog_manager.cc:1259] Loaded cluster ID: 75e0ae6c790c4b4c9ab799e19a01ae9c
I20260808 22:36:10.490568 2433 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1654884cb174485e8f06332ca4f1ebab: loading cluster ID for follower catalog manager: success
I20260808 22:36:10.492300 2433 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1654884cb174485e8f06332ca4f1ebab: acquiring CA information for follower catalog manager: success
I20260808 22:36:10.493590 2433 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1654884cb174485e8f06332ca4f1ebab: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 22:36:10.511377 2478 catalog_manager.cc:1259] Loaded cluster ID: 75e0ae6c790c4b4c9ab799e19a01ae9c
I20260808 22:36:10.511433 2478 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5595372de9c648ec9277eac84bc4fc28: loading cluster ID for follower catalog manager: success
I20260808 22:36:10.512831 2478 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5595372de9c648ec9277eac84bc4fc28: acquiring CA information for follower catalog manager: success
I20260808 22:36:10.513861 2478 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5595372de9c648ec9277eac84bc4fc28: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 22:36:10.695029 2612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:10.691176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42632 (local address 127.31.87.189:33235)
0808 22:36:10.691457 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:10.691462 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:10.691527 (+ 65us) server_negotiation.cc:407] Connection header received
0808 22:36:10.691704 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:10.691707 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:10.691836 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:10.691930 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:36:10.692138 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:10.692690 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:10.693551 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:10.693795 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:10.693910 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:10.694186 (+ 276us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:36:10.694614 (+ 428us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:36:10.694726 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 22:36:10.694825 (+ 99us) server_negotiation.cc:299] Negotiation successful
0808 22:36:10.694881 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":96,"threads_started":1}
I20260808 22:36:10.701287 2622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:10.698113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34896 (local address 127.31.87.188:37639)
0808 22:36:10.698334 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:10.698340 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:10.698355 (+ 15us) server_negotiation.cc:407] Connection header received
0808 22:36:10.698437 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:10.698439 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:10.698546 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:10.698657 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:36:10.698787 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:10.699330 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:10.700084 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:10.700294 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:10.700369 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:10.700582 (+ 213us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:36:10.700962 (+ 380us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:36:10.701049 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 22:36:10.701083 (+ 34us) server_negotiation.cc:299] Negotiation successful
0808 22:36:10.701164 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":68,"threads_started":1}
I20260808 22:36:10.707623 2612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:10.703811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42646 (local address 127.31.87.189:33235)
0808 22:36:10.703920 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:10.703924 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:10.704276 (+ 352us) server_negotiation.cc:407] Connection header received
0808 22:36:10.704429 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:10.704432 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:10.704475 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:10.704531 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:36:10.705020 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:10.705790 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:10.706642 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:10.706896 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:10.706988 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:10.707216 (+ 228us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:36:10.707293 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:36:10.707355 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 22:36:10.707439 (+ 84us) server_negotiation.cc:299] Negotiation successful
0808 22:36:10.707484 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 22:36:10.708024 2622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:10.703993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34898 (local address 127.31.87.188:37639)
0808 22:36:10.704108 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:10.704112 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:10.704411 (+ 299us) server_negotiation.cc:407] Connection header received
0808 22:36:10.704567 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:10.704570 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:10.704612 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:10.704664 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:36:10.704947 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:10.705730 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:10.706849 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:10.707122 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:10.707207 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:10.707579 (+ 372us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:36:10.707648 (+ 69us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:36:10.707691 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 22:36:10.707864 (+ 173us) server_negotiation.cc:299] Negotiation successful
0808 22:36:10.707912 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 22:36:10.708367 2632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:10.703841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58626 (local address 127.31.87.190:44685)
0808 22:36:10.704352 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:10.704356 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:10.704372 (+ 16us) server_negotiation.cc:407] Connection header received
0808 22:36:10.704466 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:10.704470 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:10.704611 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:10.704718 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:36:10.705009 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:10.705586 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:10.706791 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:10.707131 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:10.707266 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:10.707623 (+ 357us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:36:10.707909 (+ 286us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:36:10.707989 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 22:36:10.708097 (+ 108us) server_negotiation.cc:299] Negotiation successful
0808 22:36:10.708160 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":68,"threads_started":1}
I20260808 22:36:10.711088 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 2047
I20260808 22:36:10.726316 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 2180
I20260808 22:36:10.736469 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 2295
2026-08-08T22:36:10Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3197 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:36:10 dist-test-slave-266d krb5kdc[2637](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:36:10 dist-test-slave-266d krb5kdc[2637](info): set up 2 sockets
Aug 08 22:36:10 dist-test-slave-266d krb5kdc[2637](info): commencing operation
krb5kdc: starting...
W20260808 22:36:11.894645 32094 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/krb5kdc/test-user.keytab.
Aug 08 22:36:11 dist-test-slave-266d krb5kdc[2637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228571, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:36:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:36:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.87.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
Entry for principal kudu/127.31.87.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab.
I20260808 22:36:12.011957 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:39481
--webserver_interface=127.31.87.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:35699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:39481,127.31.87.189:44709,127.31.87.188:37019
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:36:12.120324 2654 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 22:36:12.120590 2654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:36:12.120631 2654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:36:12.120692 2654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:36:12.124573 2654 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:36:12.124624 2654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:36:12.124650 2654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:36:12.124671 2654 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:36:12.124701 2654 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:36:12.129693 2654 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:35699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:39481,127.31.87.189:44709,127.31.87.188:37019
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.190.keytab
--principal=kudu/127.31.87.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.190:39481
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.87.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:36:12.130607 2654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:36:12.131531 2654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:36:12.139552 2669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:36:12.139539 2662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:36:12.139734 2654 server_base.cc:1034] running on GCE node
W20260808 22:36:12.139528 2665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:36:12.140292 2654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:36:12.141355 2654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:36:12.142511 2654 hybrid_clock.cc:648] HybridClock initialized: now 1786228572142489 us; error 34 us; skew 500 ppm
Aug 08 22:36:12 dist-test-slave-266d krb5kdc[2637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228572, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:36:12.145040 2654 init.cc:377] Logged in from keytab as kudu/127.31.87.190@KRBTEST.COM (short username kudu)
I20260808 22:36:12.145900 2654 webserver.cc:460] Webserver started at http://127.31.87.190:46217/ using document root <none> and password file <none>
I20260808 22:36:12.146397 2654 fs_manager.cc:359] Metadata directory not provided
I20260808 22:36:12.146461 2654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:36:12.146673 2654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:36:12.148589 2654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/minicluster-data/master-0/data/instance:
uuid: "58723828d2fc4eac9d58875771e9e836"
format_stamp: "Formatted at 2026-08-08 22:36:12 on dist-test-slave-266d"
server_key: "7104ee1da9b6f74142e4b6e08d803aac"
server_key_iv: "69496db2731f6b61caef866f9d1dea96"
server_key_version: "encryptionkey@0"
I20260808 22:36:12.149156 2654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/minicluster-data/master-0/wal/instance:
uuid: "58723828d2fc4eac9d58875771e9e836"
format_stamp: "Formatted at 2026-08-08 22:36:12 on dist-test-slave-266d"
server_key: "7104ee1da9b6f74142e4b6e08d803aac"
server_key_iv: "69496db2731f6b61caef866f9d1dea96"
server_key_version: "encryptionkey@0"
I20260808 22:36:12.152673 2654 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.002s
I20260808 22:36:12.154886 2679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:36:12.155967 2654 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:36:12.156075 2654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/minicluster-data/master-0/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/minicluster-data/master-0/wal
uuid: "58723828d2fc4eac9d58875771e9e836"
format_stamp: "Formatted at 2026-08-08 22:36:12 on dist-test-slave-266d"
server_key: "7104ee1da9b6f74142e4b6e08d803aac"
server_key_iv: "69496db2731f6b61caef866f9d1dea96"
server_key_version: "encryptionkey@0"
I20260808 22:36:12.156157 2654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:36:12.176546 2654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:36:12.179037 2654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:36:12.179194 2654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:36:12.185671 2654 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.190:39481
I20260808 22:36:12.185946 2763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.190:39481 every 8 connection(s)
I20260808 22:36:12.186553 2654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/minicluster-data/master-0/data/info.pb
I20260808 22:36:12.187317 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 2654
I20260808 22:36:12.187439 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/minicluster-data/master-0/wal/instance
I20260808 22:36:12.187700 32094 external_mini_cluster.cc:1428] Setting key 5b2ec437839cdd6b68ce9ccaa7aa1086
I20260808 22:36:12.189251 2764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:36:12.192744 2764 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.87.190" port: 39481 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.31.87.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.189@KRBTEST.COM" created.
Aug 08 22:36:12 dist-test-slave-266d krb5kdc[2637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228572, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:36:12.206737 2768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:12.194846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:39481 (local address 127.0.0.1:51490)
0808 22:36:12.195351 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0808 22:36:12.195409 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 22:36:12.196359 (+ 950us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:36:12.196999 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:36:12.197003 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:36:12.197346 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:36:12.197593 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:36:12.197610 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:36:12.198773 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:36:12.198777 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:36:12.199468 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:36:12.199477 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:36:12.200073 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:36:12.200077 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:36:12.200171 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:12.200702 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:36:12.200718 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:36:12.202085 (+ 1367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:36:12.204107 (+ 2022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:36:12.204115 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:36:12.204123 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:36:12.204334 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:36:12.204656 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:36:12.204658 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:36:12.204660 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:36:12.204753 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:36:12.205482 (+ 729us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:36:12.205487 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:36:12.205614 (+ 127us) client_negotiation.cc:769] Sending connection context
0808 22:36:12.205755 (+ 141us) client_negotiation.cc:240] Negotiation successful
0808 22:36:12.205989 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":110,"threads_started":1}
I20260808 22:36:12.206738 2770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:12.195534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51490 (local address 127.31.87.190:39481)
0808 22:36:12.195790 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:12.195799 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:12.195835 (+ 36us) server_negotiation.cc:407] Connection header received
0808 22:36:12.196607 (+ 772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:12.196611 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:12.196831 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:12.196978 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:12.197724 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:12.198659 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:12.199576 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:12.199928 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:12.200067 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:12.202243 (+ 2176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:36:12.202251 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:36:12.202272 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:36:12.202300 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:36:12.204003 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:12.204438 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:12.204444 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:12.204446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:12.204573 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:12.204837 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:12.204840 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:12.204842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:12.205342 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:36:12.205474 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 22:36:12.205862 (+ 388us) server_negotiation.cc:299] Negotiation successful
0808 22:36:12.206071 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":60,"server-negotiator.queue_time_us":169,"thread_start_us":85,"threads_started":1}
I20260808 22:36:12.208954 2764 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.87.189" port: 44709 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:36:12.209575 2768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:12.209357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.189:44709 (local address 127.0.0.1:44238)
0808 22:36:12.209459 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0808 22:36:12.209524 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.189:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 22:36:12.209753 2685 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.87.189:44709: connect: Connection refused (error 111)
I20260808 22:36:12.210476 2768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:12.210289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.189:44709 (local address 127.0.0.1:44240)
0808 22:36:12.210386 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0808 22:36:12.210429 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.189:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 22:36:12.210613 2764 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.87.189:44709: Network error: Client connection negotiation failed: client connection to 127.31.87.189:44709: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.189.keytab.
Entry for principal kudu/127.31.87.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.189.keytab.
I20260808 22:36:12.213507 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.189:44709
--webserver_interface=127.31.87.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.189.keytab
--principal=kudu/127.31.87.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:35699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:39481,127.31.87.189:44709,127.31.87.188:37019
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 22:36:12.259826 2764 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.87.189" port: 44709 } attempt: 1
I20260808 22:36:12.260725 2768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:12.260375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.189:44709 (local address 127.0.0.1:44250)
0808 22:36:12.260544 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 22:36:12.260625 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.189:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260808 22:36:12.261474 2768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:12.261241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.189:44709 (local address 127.0.0.1:44266)
0808 22:36:12.261351 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 22:36:12.261406 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.189:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 22:36:12.261610 2764 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.87.189:44709: Network error: Client connection negotiation failed: client connection to 127.31.87.189:44709: connect: Connection refused (error 111)
W20260808 22:36:12.320123 2774 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 22:36:12.320354 2774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:36:12.320375 2774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:36:12.320432 2774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:36:12.324294 2774 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:36:12.324342 2774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:36:12.324360 2774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:36:12.324375 2774 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:36:12.324394 2774 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:36:12.329363 2774 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:35699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:39481,127.31.87.189:44709,127.31.87.188:37019
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.189.keytab
--principal=kudu/127.31.87.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.189:44709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.31.87.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2774
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:36:12.329757 2764 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.87.189" port: 44709 } attempt: 2
I20260808 22:36:12.330384 2774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:36:12.330564 2768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:12.330245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.189:44709 (local address 127.0.0.1:44270)
0808 22:36:12.330402 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 22:36:12.330477 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.189:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260808 22:36:12.331254 2768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:12.331025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.189:44709 (local address 127.0.0.1:44286)
0808 22:36:12.331138 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 22:36:12.331189 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.189:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 22:36:12.331378 2764 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.87.189:44709: Network error: Client connection negotiation failed: client connection to 127.31.87.189:44709: connect: Connection refused (error 111)
I20260808 22:36:12.331461 2774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 22:36:12.339545 2774 server_base.cc:1034] running on GCE node
W20260808 22:36:12.339581 2789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:36:12.339496 2784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:36:12.339501 2786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:36:12.340219 2774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:36:12.341230 2774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:36:12.342392 2774 hybrid_clock.cc:648] HybridClock initialized: now 1786228572342375 us; error 26 us; skew 500 ppm
Aug 08 22:36:12 dist-test-slave-266d krb5kdc[2637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228572, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:36:12.344834 2774 init.cc:377] Logged in from keytab as kudu/127.31.87.189@KRBTEST.COM (short username kudu)
I20260808 22:36:12.345691 2774 webserver.cc:460] Webserver started at http://127.31.87.189:46307/ using document root <none> and password file <none>
I20260808 22:36:12.346192 2774 fs_manager.cc:359] Metadata directory not provided
I20260808 22:36:12.346254 2774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:36:12.346454 2774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:36:12.348335 2774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/minicluster-data/master-1/data/instance:
uuid: "bf0276d9e2c74f1db32b4abc7b39a3d6"
format_stamp: "Formatted at 2026-08-08 22:36:12 on dist-test-slave-266d"
server_key: "0133505614ff14af8136c188148f3a5e"
server_key_iv: "d04f8e72883516c94b70d283aa37c547"
server_key_version: "encryptionkey@0"
I20260808 22:36:12.348894 2774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/minicluster-data/master-1/wal/instance:
uuid: "bf0276d9e2c74f1db32b4abc7b39a3d6"
format_stamp: "Formatted at 2026-08-08 22:36:12 on dist-test-slave-266d"
server_key: "0133505614ff14af8136c188148f3a5e"
server_key_iv: "d04f8e72883516c94b70d283aa37c547"
server_key_version: "encryptionkey@0"
I20260808 22:36:12.352361 2774 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 22:36:12.354575 2802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:36:12.355700 2774 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260808 22:36:12.355808 2774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/minicluster-data/master-1/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/minicluster-data/master-1/wal
uuid: "bf0276d9e2c74f1db32b4abc7b39a3d6"
format_stamp: "Formatted at 2026-08-08 22:36:12 on dist-test-slave-266d"
server_key: "0133505614ff14af8136c188148f3a5e"
server_key_iv: "d04f8e72883516c94b70d283aa37c547"
server_key_version: "encryptionkey@0"
I20260808 22:36:12.355892 2774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:36:12.385257 2774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:36:12.388029 2774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:36:12.388203 2774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:36:12.395066 2774 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.189:44709
I20260808 22:36:12.395341 2883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.189:44709 every 8 connection(s)
I20260808 22:36:12.395956 2774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/minicluster-data/master-1/data/info.pb
I20260808 22:36:12.398612 2884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:36:12.399438 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 2774
I20260808 22:36:12.399564 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/minicluster-data/master-1/wal/instance
I20260808 22:36:12.399839 32094 external_mini_cluster.cc:1428] Setting key 2b197a7c3ed53e85ab1ceba23ea51074
I20260808 22:36:12.402383 2884 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.87.190" port: 39481 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.31.87.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.87.188@KRBTEST.COM" created.
Aug 08 22:36:12 dist-test-slave-266d krb5kdc[2637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228572, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.189@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:36:12.415267 2770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:12.404169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51504 (local address 127.31.87.190:39481)
0808 22:36:12.404313 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:12.404318 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:12.405140 (+ 822us) server_negotiation.cc:407] Connection header received
0808 22:36:12.406026 (+ 886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:12.406030 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:12.406091 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:12.406191 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:12.407230 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:12.408046 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:12.409202 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:12.409431 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:12.409509 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:12.411948 (+ 2439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:36:12.411952 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:36:12.411955 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:36:12.411988 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:36:12.413364 (+ 1376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:12.413845 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:12.413848 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:12.413849 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:12.413928 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:12.414248 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:12.414251 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:12.414253 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:12.414428 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:36:12.414513 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 22:36:12.414984 (+ 471us) server_negotiation.cc:299] Negotiation successful
0808 22:36:12.415104 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 22:36:12.415985 2888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:12.404513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:39481 (local address 127.0.0.1:51504)
0808 22:36:12.404953 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0808 22:36:12.405003 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 22:36:12.405831 (+ 828us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:36:12.406314 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:36:12.406320 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:36:12.406640 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:36:12.407069 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:36:12.407081 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:36:12.408165 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:36:12.408168 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:36:12.409075 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:36:12.409115 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:36:12.409543 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:36:12.409546 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:36:12.409693 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:12.410237 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:36:12.410254 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:36:12.411781 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:36:12.413475 (+ 1694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:36:12.413480 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:36:12.413491 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:36:12.413744 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:36:12.414010 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:36:12.414013 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:36:12.414015 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:36:12.414158 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:36:12.414525 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:36:12.414531 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:36:12.414754 (+ 223us) client_negotiation.cc:769] Sending connection context
0808 22:36:12.414910 (+ 156us) client_negotiation.cc:240] Negotiation successful
0808 22:36:12.415212 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":99,"threads_started":1}
I20260808 22:36:12.417385 2884 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.87.189" port: 44709 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 22:36:12 dist-test-slave-266d krb5kdc[2637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228572, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.189@KRBTEST.COM for kudu/127.31.87.189@KRBTEST.COM
I20260808 22:36:12.422510 2764 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.87.189" port: 44709 } attempt: 3
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.87.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.188.keytab.
Entry for principal kudu/127.31.87.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.188.keytab.
I20260808 22:36:12.425891 2888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:12.417842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.189:44709 (local address 127.0.0.1:44296)
0808 22:36:12.417923 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0808 22:36:12.417933 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 22:36:12.417988 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:36:12.418498 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:36:12.418501 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:36:12.418660 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:36:12.418730 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:36:12.418735 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:36:12.419713 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:36:12.419716 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:36:12.420261 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:36:12.420269 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:36:12.420819 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:36:12.420822 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:36:12.420900 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:12.421380 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:36:12.421391 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:36:12.422478 (+ 1087us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:36:12.424415 (+ 1937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:36:12.424419 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:36:12.424421 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:36:12.424601 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:36:12.424895 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:36:12.424897 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:36:12.424899 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:36:12.424974 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:36:12.425427 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:36:12.425430 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:36:12.425518 (+ 88us) client_negotiation.cc:769] Sending connection context
0808 22:36:12.425634 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 22:36:12.425744 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260808 22:36:12.425971 32094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
/tmp/dist-test-taskOEAG_G/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.188:37019
--webserver_interface=127.31.87.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.188.keytab
--principal=kudu/127.31.87.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.87.148:35699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.87.190:39481,127.31.87.189:44709,127.31.87.188:37019
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 22:36:12.426291 2892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:12.417980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44296 (local address 127.31.87.189:44709)
0808 22:36:12.418203 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:12.418211 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:12.418225 (+ 14us) server_negotiation.cc:407] Connection header received
0808 22:36:12.418253 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:12.418256 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:12.418387 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:12.418497 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:12.418826 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:12.419600 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:12.420366 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:12.420711 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:12.420809 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:12.422595 (+ 1786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:36:12.422611 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:36:12.422622 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:36:12.422647 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:36:12.424307 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:12.424698 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:12.424704 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:12.424706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:12.424797 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:12.425057 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:12.425060 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:12.425062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:12.425324 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:36:12.425453 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 22:36:12.425715 (+ 262us) server_negotiation.cc:299] Negotiation successful
0808 22:36:12.425852 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":84,"threads_started":1}
Aug 08 22:36:12 dist-test-slave-266d krb5kdc[2637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228572, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for kudu/127.31.87.189@KRBTEST.COM
I20260808 22:36:12.428440 2884 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37019 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:36:12.429224 2888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:12.428881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.188:37019 (local address 127.0.0.1:53978)
0808 22:36:12.429050 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 22:36:12.429158 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.188:37019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
W20260808 22:36:12.429632 2804 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.87.188:37019: connect: Connection refused (error 111)
I20260808 22:36:12.430737 2888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:12.430251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.188:37019 (local address 127.0.0.1:53992)
0808 22:36:12.430625 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0808 22:36:12.430674 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.188:37019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":302}
W20260808 22:36:12.430929 2884 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.87.188:37019: Network error: Client connection negotiation failed: client connection to 127.31.87.188:37019: connect: Connection refused (error 111)
I20260808 22:36:12.431746 2768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:12.422896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.189:44709 (local address 127.0.0.1:44312)
0808 22:36:12.423005 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0808 22:36:12.423020 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:36:12.423092 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:36:12.423927 (+ 835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:36:12.423929 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:36:12.424088 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:36:12.424179 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:36:12.424185 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:36:12.425130 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:36:12.425133 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:36:12.425876 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:36:12.425887 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:36:12.426501 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:36:12.426505 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:36:12.426621 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:12.427040 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:36:12.427053 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:36:12.428379 (+ 1326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:36:12.430258 (+ 1879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:36:12.430261 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:36:12.430263 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:36:12.430473 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:36:12.430756 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:36:12.430759 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:36:12.430761 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:36:12.430853 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:36:12.431272 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:36:12.431275 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:36:12.431366 (+ 91us) client_negotiation.cc:769] Sending connection context
0808 22:36:12.431450 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 22:36:12.431578 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 22:36:12.431759 2894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:12.423227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44312 (local address 127.31.87.189:44709)
0808 22:36:12.423716 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:12.423724 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:12.423738 (+ 14us) server_negotiation.cc:407] Connection header received
0808 22:36:12.423770 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:12.423773 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:12.423810 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:12.423899 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:12.424288 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:12.424974 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:12.426016 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:12.426368 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:12.426491 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:12.428518 (+ 2027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:36:12.428522 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:36:12.428525 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:36:12.428548 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:36:12.430150 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:12.430560 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:12.430563 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:12.430565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:12.430647 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:12.430975 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:12.430977 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:12.430979 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:12.431178 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:36:12.431262 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 22:36:12.431510 (+ 248us) server_negotiation.cc:299] Negotiation successful
0808 22:36:12.431607 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":141,"threads_started":1}
I20260808 22:36:12.432431 2764 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37019 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:36:12.433108 2768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:12.432838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.188:37019 (local address 127.0.0.1:54000)
0808 22:36:12.432952 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 22:36:12.433007 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.188:37019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 22:36:12.433838 2768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:12.433618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.188:37019 (local address 127.0.0.1:54010)
0808 22:36:12.433744 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 22:36:12.433785 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.188:37019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260808 22:36:12.433959 2764 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.87.188:37019: Network error: Client connection negotiation failed: client connection to 127.31.87.188:37019: connect: Connection refused (error 111)
I20260808 22:36:12.465137 2764 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37019 } attempt: 1
I20260808 22:36:12.465905 2768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:12.465622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.188:37019 (local address 127.0.0.1:54020)
0808 22:36:12.465768 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 22:36:12.465827 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.188:37019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260808 22:36:12.466578 2768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:12.466397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.188:37019 (local address 127.0.0.1:54024)
0808 22:36:12.466479 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0808 22:36:12.466521 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.188:37019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 22:36:12.466688 2764 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.87.188:37019: Network error: Client connection negotiation failed: client connection to 127.31.87.188:37019: connect: Connection refused (error 111)
I20260808 22:36:12.480181 2884 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37019 } attempt: 1
I20260808 22:36:12.480948 2888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:12.480631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.188:37019 (local address 127.0.0.1:54028)
0808 22:36:12.480790 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 22:36:12.480865 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.188:37019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 22:36:12.481626 2888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:12.481409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.188:37019 (local address 127.0.0.1:54036)
0808 22:36:12.481505 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0808 22:36:12.481556 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.188:37019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 22:36:12.481797 2884 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.87.188:37019: Network error: Client connection negotiation failed: client connection to 127.31.87.188:37019: connect: Connection refused (error 111)
W20260808 22:36:12.534818 2895 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 22:36:12.535077 2895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:36:12.535100 2895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:36:12.535136 2895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:36:12.539042 2895 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:36:12.539093 2895 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:36:12.539111 2895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:36:12.539126 2895 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:36:12.539140 2895 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:36:12.541821 2764 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37019 } attempt: 2
I20260808 22:36:12.542629 2768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:12.542353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.188:37019 (local address 127.0.0.1:54042)
0808 22:36:12.542489 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 22:36:12.542549 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.188:37019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 22:36:12.543226 2768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:12.543051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.188:37019 (local address 127.0.0.1:54044)
0808 22:36:12.543135 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0808 22:36:12.543176 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.188:37019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260808 22:36:12.543339 2764 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.87.188:37019: Network error: Client connection negotiation failed: client connection to 127.31.87.188:37019: connect: Connection refused (error 111)
I20260808 22:36:12.544466 2895 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.87.148:35699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.31.87.190:39481,127.31.87.189:44709,127.31.87.188:37019
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/krb5kdc/kudu_127.31.87.188.keytab
--principal=kudu/127.31.87.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.87.188:37019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.31.87.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2895
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:36:12.545420 2895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:36:12.546330 2895 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 22:36:12.549954 2884 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37019 } attempt: 2
I20260808 22:36:12.550642 2888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:12.550369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.188:37019 (local address 127.0.0.1:54056)
0808 22:36:12.550513 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 22:36:12.550567 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.188:37019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260808 22:36:12.551287 2888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:12.551083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.188:37019 (local address 127.0.0.1:54072)
0808 22:36:12.551188 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0808 22:36:12.551231 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.87.188:37019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 22:36:12.551397 2884 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.87.188:37019: Network error: Client connection negotiation failed: client connection to 127.31.87.188:37019: connect: Connection refused (error 111)
W20260808 22:36:12.554353 2913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:36:12.554360 2910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:36:12.554370 2907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:36:12.554575 2895 server_base.cc:1034] running on GCE node
I20260808 22:36:12.555136 2895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:36:12.556177 2895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:36:12.557337 2895 hybrid_clock.cc:648] HybridClock initialized: now 1786228572557321 us; error 31 us; skew 500 ppm
Aug 08 22:36:12 dist-test-slave-266d krb5kdc[2637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228572, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:36:12.560029 2895 init.cc:377] Logged in from keytab as kudu/127.31.87.188@KRBTEST.COM (short username kudu)
I20260808 22:36:12.560838 2895 webserver.cc:460] Webserver started at http://127.31.87.188:42263/ using document root <none> and password file <none>
I20260808 22:36:12.561403 2895 fs_manager.cc:359] Metadata directory not provided
I20260808 22:36:12.561473 2895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:36:12.561676 2895 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:36:12.563588 2895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/minicluster-data/master-2/data/instance:
uuid: "4596e837e5174462b6c7d7c140332da7"
format_stamp: "Formatted at 2026-08-08 22:36:12 on dist-test-slave-266d"
server_key: "efb3782e8a9b55beec33837127183bae"
server_key_iv: "6cc7355a201cd4345b25a87267ff64c4"
server_key_version: "encryptionkey@0"
I20260808 22:36:12.564113 2895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/minicluster-data/master-2/wal/instance:
uuid: "4596e837e5174462b6c7d7c140332da7"
format_stamp: "Formatted at 2026-08-08 22:36:12 on dist-test-slave-266d"
server_key: "efb3782e8a9b55beec33837127183bae"
server_key_iv: "6cc7355a201cd4345b25a87267ff64c4"
server_key_version: "encryptionkey@0"
I20260808 22:36:12.567498 2895 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260808 22:36:12.569635 2924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:36:12.570737 2895 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 22:36:12.570848 2895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/minicluster-data/master-2/data,/tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/minicluster-data/master-2/wal
uuid: "4596e837e5174462b6c7d7c140332da7"
format_stamp: "Formatted at 2026-08-08 22:36:12 on dist-test-slave-266d"
server_key: "efb3782e8a9b55beec33837127183bae"
server_key_iv: "6cc7355a201cd4345b25a87267ff64c4"
server_key_version: "encryptionkey@0"
I20260808 22:36:12.570935 2895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:36:12.598310 2895 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:36:12.601271 2895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:36:12.601438 2895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:36:12.607843 2895 rpc_server.cc:307] RPC server started. Bound to: 127.31.87.188:37019
I20260808 22:36:12.608086 3022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.87.188:37019 every 8 connection(s)
I20260808 22:36:12.608721 2895 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/minicluster-data/master-2/data/info.pb
I20260808 22:36:12.611277 3025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:36:12.612216 32094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu as pid 2895
I20260808 22:36:12.612365 32094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOEAG_G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786228291013016-32094-0/minicluster-data/master-2/wal/instance
I20260808 22:36:12.612700 32094 external_mini_cluster.cc:1428] Setting key c5995204a0b17f94c619a95b0d321184
I20260808 22:36:12.615628 3025 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.87.190" port: 39481 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 22:36:12 dist-test-slave-266d krb5kdc[2637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228571, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.87.188@KRBTEST.COM
Aug 08 22:36:12 dist-test-slave-266d krb5kdc[2637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228572, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.188@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:36:12.625669 2770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:12.617908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51518 (local address 127.31.87.190:39481)
0808 22:36:12.618059 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:12.618063 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:12.618075 (+ 12us) server_negotiation.cc:407] Connection header received
0808 22:36:12.618129 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:12.618132 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:12.618179 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:12.618258 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:12.618853 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:12.619342 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:12.620329 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:12.620665 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:12.620771 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:12.622839 (+ 2068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:36:12.622842 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:36:12.622844 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:36:12.622866 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:36:12.624068 (+ 1202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:12.624514 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:12.624518 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:12.624520 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:12.624588 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:12.624827 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:12.624830 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:12.624832 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:12.624986 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:36:12.625067 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 22:36:12.625428 (+ 361us) server_negotiation.cc:299] Negotiation successful
0808 22:36:12.625540 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 22:36:12.626392 3030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:12.617381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.190:39481 (local address 127.0.0.1:51518)
0808 22:36:12.617738 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0808 22:36:12.617783 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 22:36:12.617886 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:36:12.618313 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:36:12.618317 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:36:12.618611 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:36:12.618725 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:36:12.618730 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:36:12.619451 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:36:12.619454 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:36:12.620212 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:36:12.620222 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:36:12.620797 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:36:12.620799 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:36:12.620874 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:12.621337 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:36:12.621357 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:36:12.622709 (+ 1352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:36:12.624162 (+ 1453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:36:12.624171 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:36:12.624177 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:36:12.624426 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:36:12.624668 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:36:12.624671 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:36:12.624673 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:36:12.624748 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:36:12.625080 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:36:12.625105 (+ 25us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:36:12.625227 (+ 122us) client_negotiation.cc:769] Sending connection context
0808 22:36:12.625348 (+ 121us) client_negotiation.cc:240] Negotiation successful
0808 22:36:12.625722 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":120,"thread_start_us":123,"threads_started":1}
I20260808 22:36:12.626394 3028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:12.613921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54080 (local address 127.31.87.188:37019)
0808 22:36:12.614347 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:12.614364 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:12.614406 (+ 42us) server_negotiation.cc:407] Connection header received
0808 22:36:12.615208 (+ 802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:12.615229 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:12.615636 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:12.615979 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:12.616657 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:12.618022 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:12.618922 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:12.619370 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:12.619523 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:12.622099 (+ 2576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:36:12.622113 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:36:12.622122 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:36:12.622143 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:36:12.623762 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:12.624153 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:12.624159 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:12.624165 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:12.624278 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:12.624522 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:12.624526 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:12.624528 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:12.624870 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:36:12.625066 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0808 22:36:12.625306 (+ 240us) server_negotiation.cc:299] Negotiation successful
0808 22:36:12.625650 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":279,"thread_start_us":138,"threads_started":1}
I20260808 22:36:12.627596 3025 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.87.189" port: 44709 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 22:36:12 dist-test-slave-266d krb5kdc[2637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228572, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.188@KRBTEST.COM for kudu/127.31.87.189@KRBTEST.COM
Aug 08 22:36:12 dist-test-slave-266d krb5kdc[2637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228571, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.87.190@KRBTEST.COM
I20260808 22:36:12.634827 3030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:12.627999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.189:44709 (local address 127.0.0.1:44326)
0808 22:36:12.628304 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0808 22:36:12.628316 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:36:12.628392 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:36:12.628606 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:36:12.628609 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:36:12.628725 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:36:12.628809 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:36:12.628814 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:36:12.629576 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:36:12.629580 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:36:12.630066 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:36:12.630073 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:36:12.630503 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:36:12.630506 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:36:12.630594 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:12.630968 (+ 374us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:36:12.630979 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:36:12.631990 (+ 1011us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:36:12.633508 (+ 1518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:36:12.633512 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:36:12.633514 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:36:12.633736 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:36:12.634023 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:36:12.634025 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:36:12.634026 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:36:12.634103 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:36:12.634496 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:36:12.634498 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:36:12.634569 (+ 71us) client_negotiation.cc:769] Sending connection context
0808 22:36:12.634644 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 22:36:12.634729 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245}
I20260808 22:36:12.634972 2894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:12.628191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44326 (local address 127.31.87.189:44709)
0808 22:36:12.628307 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:12.628311 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:12.628357 (+ 46us) server_negotiation.cc:407] Connection header received
0808 22:36:12.628465 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:12.628468 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:12.628518 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:12.628591 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:12.628924 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:12.629484 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:12.630166 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:12.630412 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:12.630495 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:12.632123 (+ 1628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:36:12.632127 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:36:12.632129 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:36:12.632152 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:36:12.633415 (+ 1263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:12.633843 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:12.633847 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:12.633849 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:12.633940 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:12.634188 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:12.634191 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:12.634193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:12.634406 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:36:12.634510 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 22:36:12.634718 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 22:36:12.634830 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 22:36:12.635609 3025 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37019 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:36:12.636646 2770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:12.628446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51522 (local address 127.31.87.190:39481)
0808 22:36:12.628546 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:12.628550 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:12.628793 (+ 243us) server_negotiation.cc:407] Connection header received
0808 22:36:12.628876 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:12.628879 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:12.628917 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:12.628978 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:12.629548 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:12.630008 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:12.630710 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:12.630927 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:12.631008 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:12.633404 (+ 2396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:36:12.633408 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:36:12.633410 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:36:12.633433 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:36:12.635053 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:12.635528 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:12.635530 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:12.635532 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:12.635644 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:12.635915 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:12.635918 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:12.635919 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:12.636104 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:36:12.636329 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0808 22:36:12.636449 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 22:36:12.636532 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
Aug 08 22:36:12 dist-test-slave-266d krb5kdc[2637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228572, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.188@KRBTEST.COM for kudu/127.31.87.188@KRBTEST.COM
Aug 08 22:36:12 dist-test-slave-266d krb5kdc[2637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228571, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.87.189@KRBTEST.COM
I20260808 22:36:12.642462 2764 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37019 } attempt: 3
I20260808 22:36:12.642537 2884 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37019 } attempt: 3
I20260808 22:36:12.643424 3028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:12.636051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54092 (local address 127.31.87.188:37019)
0808 22:36:12.636157 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:12.636161 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:12.636173 (+ 12us) server_negotiation.cc:407] Connection header received
0808 22:36:12.636206 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:12.636208 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:12.636244 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:12.636327 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:12.636689 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:12.637361 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:12.638109 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:12.638455 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:12.638560 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:12.640315 (+ 1755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:36:12.640319 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:36:12.640322 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:36:12.640346 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:36:12.641637 (+ 1291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:12.642047 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:12.642049 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:12.642050 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:12.642109 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:12.642352 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:12.642356 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:12.642358 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:12.642549 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:36:12.642627 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 22:36:12.643194 (+ 567us) server_negotiation.cc:299] Negotiation successful
0808 22:36:12.643292 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 22:36:12.644006 3030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:12.636005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.188:37019 (local address 127.0.0.1:54092)
0808 22:36:12.636081 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0808 22:36:12.636092 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:36:12.636152 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:36:12.636347 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:36:12.636350 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:36:12.636496 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:36:12.636575 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:36:12.636581 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:36:12.637455 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:36:12.637459 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:36:12.638019 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:36:12.638028 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:36:12.638548 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:36:12.638551 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:36:12.638604 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:12.638961 (+ 357us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:36:12.638974 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:36:12.640190 (+ 1216us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:36:12.641737 (+ 1547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:36:12.641740 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:36:12.641742 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:36:12.641963 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:36:12.642182 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:36:12.642185 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:36:12.642187 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:36:12.642264 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:36:12.642658 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:36:12.642660 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:36:12.642736 (+ 76us) client_negotiation.cc:769] Sending connection context
0808 22:36:12.642846 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 22:36:12.643879 (+ 1033us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260808 22:36:12.646090 2894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:12.637673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44330 (local address 127.31.87.189:44709)
0808 22:36:12.637771 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:12.637775 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:12.637786 (+ 11us) server_negotiation.cc:407] Connection header received
0808 22:36:12.637861 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:12.637864 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:12.637899 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:12.637961 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:12.638635 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:12.639094 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:12.640057 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:12.640265 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:12.640357 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:12.642689 (+ 2332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:36:12.642693 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:36:12.642695 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:36:12.642717 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:36:12.644295 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:12.644738 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:12.644741 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:12.644743 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:12.644835 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:12.645152 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:12.645156 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:12.645158 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:12.645347 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:36:12.645508 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 22:36:12.645846 (+ 338us) server_negotiation.cc:299] Negotiation successful
0808 22:36:12.645950 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 22:36:12.647394 3025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4596e837e5174462b6c7d7c140332da7: Bootstrap starting.
Aug 08 22:36:12 dist-test-slave-266d krb5kdc[2637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228572, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.190@KRBTEST.COM for kudu/127.31.87.188@KRBTEST.COM
Aug 08 22:36:12 dist-test-slave-266d krb5kdc[2637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228572, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.87.189@KRBTEST.COM for kudu/127.31.87.188@KRBTEST.COM
I20260808 22:36:12.650164 3025 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4596e837e5174462b6c7d7c140332da7: Neither blocks nor log segments found. Creating new log.
I20260808 22:36:12.650954 3025 log.cc:824] T 00000000000000000000000000000000 P 4596e837e5174462b6c7d7c140332da7: Log is configured to *not* fsync() on all Append() calls
I20260808 22:36:12.652420 2768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:12.642804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.188:37019 (local address 127.0.0.1:54104)
0808 22:36:12.643029 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0808 22:36:12.643040 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:36:12.643098 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:36:12.644234 (+ 1136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:36:12.644237 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:36:12.644393 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:36:12.644486 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:36:12.644493 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:36:12.645478 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:36:12.645481 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:36:12.646223 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:36:12.646234 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:36:12.646820 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:36:12.646824 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:36:12.646901 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:12.647357 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:36:12.647372 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:36:12.648548 (+ 1176us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:36:12.650534 (+ 1986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:36:12.650537 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:36:12.650539 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:36:12.650789 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:36:12.651082 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:36:12.651086 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:36:12.651088 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:36:12.651174 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:36:12.651945 (+ 771us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:36:12.651948 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:36:12.652041 (+ 93us) client_negotiation.cc:769] Sending connection context
0808 22:36:12.652142 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 22:36:12.652276 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165}
I20260808 22:36:12.652659 2888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:12.642804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.87.188:37019 (local address 127.0.0.1:54112)
0808 22:36:12.642948 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 22:36:12.642960 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:36:12.643027 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:36:12.643760 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:36:12.643763 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:36:12.643912 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:36:12.643983 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:36:12.643987 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:36:12.646524 (+ 2537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:36:12.646527 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:36:12.647258 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:36:12.647269 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:36:12.647798 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:36:12.647802 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:36:12.647878 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:12.648294 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:36:12.648307 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:36:12.649598 (+ 1291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:36:12.651265 (+ 1667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:36:12.651268 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:36:12.651270 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:36:12.651499 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:36:12.651754 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:36:12.651757 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:36:12.651759 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:36:12.651839 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:36:12.652219 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:36:12.652222 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:36:12.652312 (+ 90us) client_negotiation.cc:769] Sending connection context
0808 22:36:12.652395 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 22:36:12.652510 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83}
I20260808 22:36:12.653040 3036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:12.643209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54104 (local address 127.31.87.188:37019)
0808 22:36:12.643629 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:12.643638 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:12.643653 (+ 15us) server_negotiation.cc:407] Connection header received
0808 22:36:12.643685 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:12.643688 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:12.644103 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:12.644598 (+ 495us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:12.644700 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:12.645362 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:12.646363 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:12.646697 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:12.647828 (+ 1131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:12.648673 (+ 845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:36:12.648676 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:36:12.648679 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:36:12.648703 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:36:12.650415 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:12.650892 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:12.650896 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:12.650897 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:12.650989 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:12.651614 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:12.651617 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:12.651619 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:12.651820 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:36:12.651948 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 22:36:12.652787 (+ 839us) server_negotiation.cc:299] Negotiation successful
0808 22:36:12.652891 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":90,"threads_started":1}
I20260808 22:36:12.653517 3028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:12.643395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54112 (local address 127.31.87.188:37019)
0808 22:36:12.643572 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:12.643576 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:12.643590 (+ 14us) server_negotiation.cc:407] Connection header received
0808 22:36:12.643622 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:12.643625 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:12.643663 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:12.643743 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:12.645667 (+ 1924us) spinlock_profiling.cc:243] Waited 1.37 ms on lock 0x2f5b7c88. stack: 00007f87967aac78 00007f87967ab1a9 00007f879636d1d7 00000000005984a6 00000000005984f9 000000000059c488 00007f87967a72ab 00007f87967a69da 00007f879ef04817 00007f879ef1707f 00007f879ae90c99 00007f879ae85e9d 00007f879ae3f219 00007f879ae3f615 00007f879ae550d3 00007f879ae5854b
0808 22:36:12.645748 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:12.646399 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:12.647398 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:12.647664 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:12.648061 (+ 397us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:12.649693 (+ 1632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:36:12.649697 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:36:12.649699 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:36:12.649723 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:36:12.651163 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:12.651578 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:12.651581 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:12.651583 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:12.651672 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:12.651913 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:12.651916 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:12.651918 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:12.652105 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:36:12.653076 (+ 971us) server_negotiation.cc:1035] Waiting for connection context
0808 22:36:12.653175 (+ 99us) server_negotiation.cc:299] Negotiation successful
0808 22:36:12.653378 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":125,"spinlock_wait_cycles":3154688,"thread_start_us":84,"threads_started":1}
I20260808 22:36:12.653662 32094 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:36:12.653730 3025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4596e837e5174462b6c7d7c140332da7: No bootstrap required, opened a new log
I20260808 22:36:12.654827 2718 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "58723828d2fc4eac9d58875771e9e836"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51522
I20260808 22:36:12.656128 2764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58723828d2fc4eac9d58875771e9e836: Bootstrap starting.
I20260808 22:36:12.656443 2884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf0276d9e2c74f1db32b4abc7b39a3d6: Bootstrap starting.
I20260808 22:36:12.657697 3025 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4596e837e5174462b6c7d7c140332da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58723828d2fc4eac9d58875771e9e836" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 39481 } } peers { permanent_uuid: "bf0276d9e2c74f1db32b4abc7b39a3d6" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 44709 } } peers { permanent_uuid: "4596e837e5174462b6c7d7c140332da7" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37019 } }
I20260808 22:36:12.657941 3025 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4596e837e5174462b6c7d7c140332da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4596e837e5174462b6c7d7c140332da7, State: Initialized, Role: FOLLOWER
I20260808 22:36:12.658262 3025 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4596e837e5174462b6c7d7c140332da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58723828d2fc4eac9d58875771e9e836" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 39481 } } peers { permanent_uuid: "bf0276d9e2c74f1db32b4abc7b39a3d6" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 44709 } } peers { permanent_uuid: "4596e837e5174462b6c7d7c140332da7" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37019 } }
I20260808 22:36:12.658648 2764 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 58723828d2fc4eac9d58875771e9e836: Neither blocks nor log segments found. Creating new log.
I20260808 22:36:12.659008 2884 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf0276d9e2c74f1db32b4abc7b39a3d6: Neither blocks nor log segments found. Creating new log.
I20260808 22:36:12.659487 2764 log.cc:824] T 00000000000000000000000000000000 P 58723828d2fc4eac9d58875771e9e836: Log is configured to *not* fsync() on all Append() calls
I20260808 22:36:12.659543 3025 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4596e837e5174462b6c7d7c140332da7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:36:12.659823 2884 log.cc:824] T 00000000000000000000000000000000 P bf0276d9e2c74f1db32b4abc7b39a3d6: Log is configured to *not* fsync() on all Append() calls
I20260808 22:36:12.661837 2764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58723828d2fc4eac9d58875771e9e836: No bootstrap required, opened a new log
I20260808 22:36:12.661836 2884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf0276d9e2c74f1db32b4abc7b39a3d6: No bootstrap required, opened a new log
I20260808 22:36:12.664716 2764 raft_consensus.cc:348] T 00000000000000000000000000000000 P 58723828d2fc4eac9d58875771e9e836 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58723828d2fc4eac9d58875771e9e836" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 39481 } } peers { permanent_uuid: "bf0276d9e2c74f1db32b4abc7b39a3d6" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 44709 } } peers { permanent_uuid: "4596e837e5174462b6c7d7c140332da7" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37019 } }
I20260808 22:36:12.664927 2764 raft_consensus.cc:729] T 00000000000000000000000000000000 P 58723828d2fc4eac9d58875771e9e836 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58723828d2fc4eac9d58875771e9e836, State: Initialized, Role: FOLLOWER
I20260808 22:36:12.661586 3040 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4596e837e5174462b6c7d7c140332da7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58723828d2fc4eac9d58875771e9e836" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 39481 } } peers { permanent_uuid: "bf0276d9e2c74f1db32b4abc7b39a3d6" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 44709 } } peers { permanent_uuid: "4596e837e5174462b6c7d7c140332da7" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37019 } } }
I20260808 22:36:12.665252 2764 consensus_queue.cc:260] T 00000000000000000000000000000000 P 58723828d2fc4eac9d58875771e9e836 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58723828d2fc4eac9d58875771e9e836" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 39481 } } peers { permanent_uuid: "bf0276d9e2c74f1db32b4abc7b39a3d6" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 44709 } } peers { permanent_uuid: "4596e837e5174462b6c7d7c140332da7" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37019 } }
I20260808 22:36:12.665517 3040 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4596e837e5174462b6c7d7c140332da7 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:36:12.665608 2884 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf0276d9e2c74f1db32b4abc7b39a3d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58723828d2fc4eac9d58875771e9e836" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 39481 } } peers { permanent_uuid: "bf0276d9e2c74f1db32b4abc7b39a3d6" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 44709 } } peers { permanent_uuid: "4596e837e5174462b6c7d7c140332da7" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37019 } }
I20260808 22:36:12.666599 2764 sys_catalog.cc:564] T 00000000000000000000000000000000 P 58723828d2fc4eac9d58875771e9e836 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:36:12.666656 2718 raft_consensus.cc:482] T 00000000000000000000000000000000 P 58723828d2fc4eac9d58875771e9e836 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 22:36:12.666766 2718 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 58723828d2fc4eac9d58875771e9e836 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:36:12.667902 2718 raft_consensus.cc:504] T 00000000000000000000000000000000 P 58723828d2fc4eac9d58875771e9e836 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58723828d2fc4eac9d58875771e9e836" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 39481 } } peers { permanent_uuid: "bf0276d9e2c74f1db32b4abc7b39a3d6" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 44709 } } peers { permanent_uuid: "4596e837e5174462b6c7d7c140332da7" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37019 } }
I20260808 22:36:12.666679 2895 master_runner.cc:186] Error getting master registration for 127.31.87.190:39481: OK, instance_id { permanent_uuid: "58723828d2fc4eac9d58875771e9e836" instance_seqno: 1786228572183871 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 22:36:12.669902 2895 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 22:36:12.665808 2884 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf0276d9e2c74f1db32b4abc7b39a3d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf0276d9e2c74f1db32b4abc7b39a3d6, State: Initialized, Role: FOLLOWER
I20260808 22:36:12.668833 2718 leader_election.cc:288] T 00000000000000000000000000000000 P 58723828d2fc4eac9d58875771e9e836 [CANDIDATE]: Term 1 election: Requested vote from peers bf0276d9e2c74f1db32b4abc7b39a3d6 (127.31.87.189:44709), 4596e837e5174462b6c7d7c140332da7 (127.31.87.188:37019)
I20260808 22:36:12.671319 2973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "58723828d2fc4eac9d58875771e9e836" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "4596e837e5174462b6c7d7c140332da7"
I20260808 22:36:12.671541 2973 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4596e837e5174462b6c7d7c140332da7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:36:12.672847 2973 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4596e837e5174462b6c7d7c140332da7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58723828d2fc4eac9d58875771e9e836 in term 1.
I20260808 22:36:12.673286 2682 leader_election.cc:302] T 00000000000000000000000000000000 P 58723828d2fc4eac9d58875771e9e836 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4596e837e5174462b6c7d7c140332da7, 58723828d2fc4eac9d58875771e9e836; no voters:
W20260808 22:36:12.664146 3063 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4596e837e5174462b6c7d7c140332da7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:36:12.674096 3063 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260808 22:36:12.674280 3083 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 58723828d2fc4eac9d58875771e9e836: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:36:12.674355 3083 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:36:12.675232 2654 master_runner.cc:186] Error getting master registration for 127.31.87.189:44709: OK, instance_id { permanent_uuid: "bf0276d9e2c74f1db32b4abc7b39a3d6" instance_seqno: 1786228572393161 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 22:36:12.678375 2654 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 22:36:12.678730 2884 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf0276d9e2c74f1db32b4abc7b39a3d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58723828d2fc4eac9d58875771e9e836" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 39481 } } peers { permanent_uuid: "bf0276d9e2c74f1db32b4abc7b39a3d6" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 44709 } } peers { permanent_uuid: "4596e837e5174462b6c7d7c140332da7" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37019 } }
W20260808 22:36:12.680474 2688 tablet.cc:2369] T 00000000000000000000000000000000 P 58723828d2fc4eac9d58875771e9e836: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:36:12.675204 3090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58723828d2fc4eac9d58875771e9e836 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58723828d2fc4eac9d58875771e9e836" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 39481 } } peers { permanent_uuid: "bf0276d9e2c74f1db32b4abc7b39a3d6" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 44709 } } peers { permanent_uuid: "4596e837e5174462b6c7d7c140332da7" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37019 } } }
I20260808 22:36:12.681246 3090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58723828d2fc4eac9d58875771e9e836 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:36:12.681339 3090 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 58723828d2fc4eac9d58875771e9e836 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:36:12.681416 3090 raft_consensus.cc:686] T 00000000000000000000000000000000 P 58723828d2fc4eac9d58875771e9e836 [term 1 LEADER]: Becoming Leader. State: Replica: 58723828d2fc4eac9d58875771e9e836, State: Running, Role: LEADER
I20260808 22:36:12.681483 2842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "58723828d2fc4eac9d58875771e9e836" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "bf0276d9e2c74f1db32b4abc7b39a3d6"
I20260808 22:36:12.681695 2842 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf0276d9e2c74f1db32b4abc7b39a3d6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:36:12.681726 3090 consensus_queue.cc:237] T 00000000000000000000000000000000 P 58723828d2fc4eac9d58875771e9e836 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58723828d2fc4eac9d58875771e9e836" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 39481 } } peers { permanent_uuid: "bf0276d9e2c74f1db32b4abc7b39a3d6" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 44709 } } peers { permanent_uuid: "4596e837e5174462b6c7d7c140332da7" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37019 } }
I20260808 22:36:12.683586 2842 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bf0276d9e2c74f1db32b4abc7b39a3d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58723828d2fc4eac9d58875771e9e836 in term 1.
I20260808 22:36:12.683662 3090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58723828d2fc4eac9d58875771e9e836 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 58723828d2fc4eac9d58875771e9e836. Latest consensus state: current_term: 1 leader_uuid: "58723828d2fc4eac9d58875771e9e836" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58723828d2fc4eac9d58875771e9e836" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 39481 } } peers { permanent_uuid: "bf0276d9e2c74f1db32b4abc7b39a3d6" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 44709 } } peers { permanent_uuid: "4596e837e5174462b6c7d7c140332da7" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37019 } } }
I20260808 22:36:12.683779 3090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58723828d2fc4eac9d58875771e9e836 [sys.catalog]: This master's current role is: LEADER
I20260808 22:36:12.681416 2884 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf0276d9e2c74f1db32b4abc7b39a3d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:36:12.684401 3028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:12.673590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54116 (local address 127.31.87.188:37019)
0808 22:36:12.673705 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:12.673710 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:12.675177 (+ 1467us) server_negotiation.cc:407] Connection header received
0808 22:36:12.675360 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:12.675364 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:12.675409 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:12.675457 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:12.676417 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:12.677224 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:12.678430 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:12.678755 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:12.678824 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:12.680704 (+ 1880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:36:12.680709 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:36:12.680712 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:36:12.680737 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:36:12.682501 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:12.682966 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:12.682970 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:12.682972 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:12.683049 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:12.683344 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:12.683347 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:12.683349 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:12.683543 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:36:12.683636 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 22:36:12.684124 (+ 488us) server_negotiation.cc:299] Negotiation successful
0808 22:36:12.684233 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260808 22:36:12.690641 3117 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bf0276d9e2c74f1db32b4abc7b39a3d6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:36:12.690716 3117 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:36:12.693877 2894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:12.674605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44340 (local address 127.31.87.189:44709)
0808 22:36:12.674944 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:12.674949 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:12.685420 (+ 10471us) server_negotiation.cc:407] Connection header received
0808 22:36:12.685560 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:12.685563 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:12.685608 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:12.685654 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:12.686405 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:12.687166 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:12.688427 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:12.688753 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:12.688823 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:12.690434 (+ 1611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:36:12.690438 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:36:12.690441 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:36:12.690466 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:36:12.692090 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:12.692577 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:12.692580 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:12.692583 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:12.692663 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:12.692942 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:12.692946 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:12.692948 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:12.693167 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:36:12.693277 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 22:36:12.693623 (+ 346us) server_negotiation.cc:299] Negotiation successful
0808 22:36:12.693730 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269}
I20260808 22:36:12.695109 3098 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:36:12.696064 3098 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:36:12.698052 2770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:12.674052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51532 (local address 127.31.87.190:39481)
0808 22:36:12.674162 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:12.674166 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:12.685325 (+ 11159us) server_negotiation.cc:407] Connection header received
0808 22:36:12.685493 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:12.685496 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:12.685546 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:12.685591 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:12.686477 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:12.687200 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:12.688427 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:12.688753 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:12.688824 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:12.694485 (+ 5661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:36:12.694489 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:36:12.694491 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:36:12.694515 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:36:12.696089 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:12.696493 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:12.696497 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:12.696498 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:12.696580 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:36:12.696853 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:36:12.696856 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:36:12.696857 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:36:12.697047 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:36:12.697219 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0808 22:36:12.697805 (+ 586us) server_negotiation.cc:299] Negotiation successful
0808 22:36:12.697911 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 22:36:12.698522 3096 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf0276d9e2c74f1db32b4abc7b39a3d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58723828d2fc4eac9d58875771e9e836" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 39481 } } peers { permanent_uuid: "bf0276d9e2c74f1db32b4abc7b39a3d6" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 44709 } } peers { permanent_uuid: "4596e837e5174462b6c7d7c140332da7" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37019 } } }
I20260808 22:36:12.698652 3096 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf0276d9e2c74f1db32b4abc7b39a3d6 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:36:12.702875 2973 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4596e837e5174462b6c7d7c140332da7 [term 1 FOLLOWER]: Refusing update from remote peer 58723828d2fc4eac9d58875771e9e836: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:36:12.702914 2842 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bf0276d9e2c74f1db32b4abc7b39a3d6 [term 1 FOLLOWER]: Refusing update from remote peer 58723828d2fc4eac9d58875771e9e836: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:36:12.703491 3066 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 58723828d2fc4eac9d58875771e9e836 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf0276d9e2c74f1db32b4abc7b39a3d6" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 44709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:36:12.703920 3066 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 58723828d2fc4eac9d58875771e9e836 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4596e837e5174462b6c7d7c140332da7" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:36:12.708758 3096 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf0276d9e2c74f1db32b4abc7b39a3d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 58723828d2fc4eac9d58875771e9e836. Latest consensus state: current_term: 1 leader_uuid: "58723828d2fc4eac9d58875771e9e836" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58723828d2fc4eac9d58875771e9e836" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 39481 } } peers { permanent_uuid: "bf0276d9e2c74f1db32b4abc7b39a3d6" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 44709 } } peers { permanent_uuid: "4596e837e5174462b6c7d7c140332da7" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37019 } } }
I20260808 22:36:12.708863 3096 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf0276d9e2c74f1db32b4abc7b39a3d6 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:36:12.709784 3090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58723828d2fc4eac9d58875771e9e836 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "58723828d2fc4eac9d58875771e9e836" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58723828d2fc4eac9d58875771e9e836" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 39481 } } peers { permanent_uuid: "bf0276d9e2c74f1db32b4abc7b39a3d6" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 44709 } } peers { permanent_uuid: "4596e837e5174462b6c7d7c140332da7" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37019 } } }
I20260808 22:36:12.709887 3090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58723828d2fc4eac9d58875771e9e836 [sys.catalog]: This master's current role is: LEADER
I20260808 22:36:12.710458 3096 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf0276d9e2c74f1db32b4abc7b39a3d6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "58723828d2fc4eac9d58875771e9e836" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58723828d2fc4eac9d58875771e9e836" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 39481 } } peers { permanent_uuid: "bf0276d9e2c74f1db32b4abc7b39a3d6" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 44709 } } peers { permanent_uuid: "4596e837e5174462b6c7d7c140332da7" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37019 } } }
I20260808 22:36:12.710556 3096 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf0276d9e2c74f1db32b4abc7b39a3d6 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:36:12.710971 3120 mvcc.cc:205] Tried to move back new op lower bound from 7316392233784684544 to 7316392233708539904. Current Snapshot: MvccSnapshot[applied={T|T < 7316392233784684544}]
I20260808 22:36:12.711843 3040 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4596e837e5174462b6c7d7c140332da7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 58723828d2fc4eac9d58875771e9e836. Latest consensus state: current_term: 1 leader_uuid: "58723828d2fc4eac9d58875771e9e836" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58723828d2fc4eac9d58875771e9e836" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 39481 } } peers { permanent_uuid: "bf0276d9e2c74f1db32b4abc7b39a3d6" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 44709 } } peers { permanent_uuid: "4596e837e5174462b6c7d7c140332da7" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37019 } } }
I20260808 22:36:12.711961 3040 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4596e837e5174462b6c7d7c140332da7 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:36:12.713479 3066 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58723828d2fc4eac9d58875771e9e836 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "58723828d2fc4eac9d58875771e9e836" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58723828d2fc4eac9d58875771e9e836" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 39481 } } peers { permanent_uuid: "bf0276d9e2c74f1db32b4abc7b39a3d6" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 44709 } } peers { permanent_uuid: "4596e837e5174462b6c7d7c140332da7" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37019 } } }
I20260808 22:36:12.713588 3066 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58723828d2fc4eac9d58875771e9e836 [sys.catalog]: This master's current role is: LEADER
I20260808 22:36:12.714473 3040 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4596e837e5174462b6c7d7c140332da7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "58723828d2fc4eac9d58875771e9e836" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58723828d2fc4eac9d58875771e9e836" member_type: VOTER last_known_addr { host: "127.31.87.190" port: 39481 } } peers { permanent_uuid: "bf0276d9e2c74f1db32b4abc7b39a3d6" member_type: VOTER last_known_addr { host: "127.31.87.189" port: 44709 } } peers { permanent_uuid: "4596e837e5174462b6c7d7c140332da7" member_type: VOTER last_known_addr { host: "127.31.87.188" port: 37019 } } }
I20260808 22:36:12.714536 3122 mvcc.cc:205] Tried to move back new op lower bound from 7316392233784684544 to 7316392233708539904. Current Snapshot: MvccSnapshot[applied={T|T < 7316392233784684544}]
I20260808 22:36:12.714576 3040 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4596e837e5174462b6c7d7c140332da7 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:36:12.718654 3098 catalog_manager.cc:1347] Generated new cluster ID: 4001c09876d8480a86d9a597217fd1ab
I20260808 22:36:12.718716 3098 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:36:12.722728 3127 mvcc.cc:205] Tried to move back new op lower bound from 7316392233784684544 to 7316392233708539904. Current Snapshot: MvccSnapshot[applied={T|T < 7316392233784684544}]
I20260808 22:36:12.757813 3098 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:36:12.759172 3098 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:36:12.789426 3098 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 58723828d2fc4eac9d58875771e9e836: Generated new TSK 0
I20260808 22:36:12.790391 3098 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 22:36:12.816969 2770 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:36:12.814238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51534 (local address 127.31.87.190:39481)
0808 22:36:12.814355 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:12.814360 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:12.815520 (+ 1160us) server_negotiation.cc:407] Connection header received
0808 22:36:12.816091 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:12.816095 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:12.816150 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:12.816202 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:12.816840 (+ 638us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51534: BlockingRecv error: recv got EOF from 127.0.0.1:51534 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260808 22:36:12.818825 2894 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:36:12.814919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44348 (local address 127.31.87.189:44709)
0808 22:36:12.815029 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:12.815033 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:12.817616 (+ 2583us) server_negotiation.cc:407] Connection header received
0808 22:36:12.817929 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:12.817932 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:12.817983 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:12.818031 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:12.818713 (+ 682us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44348: BlockingRecv error: recv got EOF from 127.0.0.1:44348 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260808 22:36:12.819832 3028 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:36:12.815965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54122 (local address 127.31.87.188:37019)
0808 22:36:12.816092 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:12.816097 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:12.818255 (+ 2158us) server_negotiation.cc:407] Connection header received
0808 22:36:12.819142 (+ 887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:12.819146 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:12.819193 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:12.819239 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:12.819724 (+ 485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54122: BlockingRecv error: recv got EOF from 127.0.0.1:54122 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260808 22:36:12.823940 2894 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:36:12.822530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44352 (local address 127.31.87.189:44709)
0808 22:36:12.822634 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:12.822638 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:12.823080 (+ 442us) server_negotiation.cc:407] Connection header received
0808 22:36:12.823275 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:12.823278 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:12.823343 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:12.823389 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:12.823855 (+ 466us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44352: BlockingRecv error: recv got EOF from 127.0.0.1:44352 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260808 22:36:12.828540 2894 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:36:12.827186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44360 (local address 127.31.87.189:44709)
0808 22:36:12.827278 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:12.827281 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:12.827771 (+ 490us) server_negotiation.cc:407] Connection header received
0808 22:36:12.827931 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:12.827934 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:12.827979 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:12.828025 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:12.828462 (+ 437us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44360: BlockingRecv error: recv got EOF from 127.0.0.1:44360 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260808 22:36:12.833781 2894 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:36:12.832304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44370 (local address 127.31.87.189:44709)
0808 22:36:12.832395 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:12.832399 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:12.832904 (+ 505us) server_negotiation.cc:407] Connection header received
0808 22:36:12.833076 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:12.833079 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:12.833177 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:12.833223 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:12.833700 (+ 477us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44370: BlockingRecv error: recv got EOF from 127.0.0.1:44370 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260808 22:36:12.840972 2894 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:36:12.839528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44386 (local address 127.31.87.189:44709)
0808 22:36:12.839663 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:12.839667 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:12.840156 (+ 489us) server_negotiation.cc:407] Connection header received
0808 22:36:12.840322 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:12.840325 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:12.840370 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:12.840413 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:12.840888 (+ 475us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44386: BlockingRecv error: recv got EOF from 127.0.0.1:44386 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260808 22:36:12.852754 2894 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:36:12.851078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44400 (local address 127.31.87.189:44709)
0808 22:36:12.851193 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:12.851197 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:12.851785 (+ 588us) server_negotiation.cc:407] Connection header received
0808 22:36:12.851991 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:12.851995 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:12.852047 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:12.852099 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:12.852653 (+ 554us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44400: BlockingRecv error: recv got EOF from 127.0.0.1:44400 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260808 22:36:12.872179 2894 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:36:12.870960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44416 (local address 127.31.87.189:44709)
0808 22:36:12.871067 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:12.871071 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:12.871342 (+ 271us) server_negotiation.cc:407] Connection header received
0808 22:36:12.871448 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:12.871451 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:12.871513 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:12.871602 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:12.872067 (+ 465us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44416: BlockingRecv error: recv got EOF from 127.0.0.1:44416 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260808 22:36:12.907074 2894 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:36:12.906080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44426 (local address 127.31.87.189:44709)
0808 22:36:12.906232 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:12.906237 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:12.906350 (+ 113us) server_negotiation.cc:407] Connection header received
0808 22:36:12.906466 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:12.906469 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:12.906517 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:12.906587 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:12.906955 (+ 368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44426: BlockingRecv error: recv got EOF from 127.0.0.1:44426 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260808 22:36:12.974145 2894 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:36:12.972980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44434 (local address 127.31.87.189:44709)
0808 22:36:12.973155 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:12.973160 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:12.973285 (+ 125us) server_negotiation.cc:407] Connection header received
0808 22:36:12.973433 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:12.973436 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:12.973488 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:12.973556 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:12.974019 (+ 463us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44434: BlockingRecv error: recv got EOF from 127.0.0.1:44434 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260808 22:36:13.105242 2894 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:36:13.104161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44438 (local address 127.31.87.189:44709)
0808 22:36:13.104305 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:13.104309 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:13.104443 (+ 134us) server_negotiation.cc:407] Connection header received
0808 22:36:13.104567 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:13.104570 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:13.104634 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:13.104706 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:13.105118 (+ 412us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44438: BlockingRecv error: recv got EOF from 127.0.0.1:44438 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260808 22:36:13.364207 2894 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:36:13.363236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44446 (local address 127.31.87.189:44709)
0808 22:36:13.363391 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:13.363395 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:13.363450 (+ 55us) server_negotiation.cc:407] Connection header received
0808 22:36:13.363569 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:13.363572 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:13.363623 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:13.363693 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:36:13.364061 (+ 368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44446: BlockingRecv error: recv got EOF from 127.0.0.1:44446 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 22:36:13.675479 3063 catalog_manager.cc:1259] Loaded cluster ID: 4001c09876d8480a86d9a597217fd1ab
I20260808 22:36:13.675558 3063 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4596e837e5174462b6c7d7c140332da7: loading cluster ID for follower catalog manager: success
I20260808 22:36:13.677479 3063 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4596e837e5174462b6c7d7c140332da7: acquiring CA information for follower catalog manager: success
I20260808 22:36:13.678689 3063 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4596e837e5174462b6c7d7c140332da7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 22:36:13.691864 3117 catalog_manager.cc:1259] Loaded cluster ID: 4001c09876d8480a86d9a597217fd1ab
I20260808 22:36:13.691927 3117 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bf0276d9e2c74f1db32b4abc7b39a3d6: loading cluster ID for follower catalog manager: success
I20260808 22:36:13.693521 3117 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bf0276d9e2c74f1db32b4abc7b39a3d6: acquiring CA information for follower catalog manager: success
I20260808 22:36:13.694597 3117 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bf0276d9e2c74f1db32b4abc7b39a3d6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 22:36:13.882001 3252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:13.878122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44454 (local address 127.31.87.189:44709)
0808 22:36:13.878386 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:13.878390 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:13.878406 (+ 16us) server_negotiation.cc:407] Connection header received
0808 22:36:13.878662 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:13.878667 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:13.878825 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:13.878942 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:36:13.879156 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:13.879711 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:13.880507 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:13.880771 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:13.880855 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:13.881136 (+ 281us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:36:13.881612 (+ 476us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:36:13.881711 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 22:36:13.881761 (+ 50us) server_negotiation.cc:299] Negotiation successful
0808 22:36:13.881813 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":73,"threads_started":1}
I20260808 22:36:13.888274 3264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:13.884888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54136 (local address 127.31.87.188:37019)
0808 22:36:13.885084 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:13.885136 (+ 52us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:13.885152 (+ 16us) server_negotiation.cc:407] Connection header received
0808 22:36:13.885268 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:13.885271 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:13.885425 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:13.885553 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:36:13.885720 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:13.886221 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:13.887042 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:13.887280 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:13.887356 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:13.887553 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:36:13.887952 (+ 399us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:36:13.888038 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 22:36:13.888063 (+ 25us) server_negotiation.cc:299] Negotiation successful
0808 22:36:13.888119 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":65,"threads_started":1}
I20260808 22:36:13.895104 3275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:13.890977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51548 (local address 127.31.87.190:39481)
0808 22:36:13.891483 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:13.891486 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:13.891497 (+ 11us) server_negotiation.cc:407] Connection header received
0808 22:36:13.891568 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:13.891570 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:13.891672 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:13.891778 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:36:13.892000 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:13.892754 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:13.893756 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:13.893988 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:13.894077 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:13.894345 (+ 268us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:36:13.894595 (+ 250us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:36:13.894675 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 22:36:13.894754 (+ 79us) server_negotiation.cc:299] Negotiation successful
0808 22:36:13.894818 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":67,"threads_started":1}
I20260808 22:36:13.895784 3252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:13.891266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44462 (local address 127.31.87.189:44709)
0808 22:36:13.891361 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:13.891365 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:13.892252 (+ 887us) server_negotiation.cc:407] Connection header received
0808 22:36:13.892401 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:13.892404 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:13.892454 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:13.892522 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:36:13.892701 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:13.893262 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:13.894609 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:13.894918 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:13.895011 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:13.895319 (+ 308us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:36:13.895393 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:36:13.895439 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 22:36:13.895612 (+ 173us) server_negotiation.cc:299] Negotiation successful
0808 22:36:13.895665 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 22:36:13.895972 3264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:36:13.891045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54144 (local address 127.31.87.188:37019)
0808 22:36:13.891428 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0808 22:36:13.891432 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:36:13.892053 (+ 621us) server_negotiation.cc:407] Connection header received
0808 22:36:13.892133 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:36:13.892136 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:36:13.892184 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:36:13.892260 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:36:13.892504 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:13.893216 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:13.894633 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:36:13.894933 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:36:13.895137 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:36:13.895375 (+ 238us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:36:13.895450 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:36:13.895780 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0808 22:36:13.895814 (+ 34us) server_negotiation.cc:299] Negotiation successful
0808 22:36:13.895864 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 22:36:13.898124 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 2654
I20260808 22:36:13.910976 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 2774
I20260808 22:36:13.925053 32094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOEAG_G/build/debug/bin/kudu with pid 2895
2026-08-08T22:36:13Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3187 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6384 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (282891 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST